How to write clean functional code in Scala and keep it that way

Most of the development we do at Newsweaver has been heavily Java-based so far. In the last year, we started working on Spark applications and hence jumped into Scala. This is a brief summary of some guidelines we wish we had when we started and how we are relying on the Scala WartRemover tool to keep our code clean and in line with functional concepts.

Functional Programming and immutability

Immutability allows for clearer code. When objects are not changing, there is no need to track their state. As a result, functional code reads like a combination of transformations, rather than a step by step list of instructions.

Concepts like immutability are ingrained in Functional Programming but they can be pretty foreign to developers coming from a heavy imperative programming background. Scala allows for doing both, so it is easy to keep writing imperative code and miss out on all the functional goodness.

Here are some basic guidelines on how to write clean functional code and avoid imperative concepts from creeping in.

Stay away from mutable variables

Scala allows to declare identifiers with val (immutable value) or var (mutable variable). In Java, variables are all mutable unless explicitly marked as final - making the transition can be a bit disturbing, for example when trying to reuse a variable in a loop.

Example: Calculating the sum of all elements in a list

Bad: Keep adding to the same variable

val list = List(1,1,2,3)

var sum = 0  
list.foreach { i => sum = sum+i }  

Better: Use fold

val sum = list.foldLeft(0)(_+_)  

...or reduce

val sum = list.reduce(_+_)  

Note: We can also use list.sum, but what is the fun in that?

Only use immutable collections

By default, all collections in Scala are immutable. For example, when defining a List, it is not possible to add, remove or reorder elements in the instance of that List. Instead, operations on the List are returning a new List as a result.

Scala provides mutable versions of the collections as well and the temptation is to use these instead, especially when coming from Java where we would do this every day. Using immutable collections will force you to think in a more functional way when solving a problem. Sometimes, you don’t even need to use a collection after all!

Example: Count the number of occurrences of each number in a list

Bad: Using a mutable map

val map = scala.collection.mutable.Map[Int, Int]()  
list.foreach { i => map + ((i, map.getOrElse(i, 0)+1)) }  

Better: Using fold

val map = list.foldLeft(Map[Int, Int]()) {  
      (map, i) => map + ((i, map.getOrElse(i,0)+1))

Even Better: Using group by

val map = list.groupBy(identity).mapValues(_.size)  

Ban null, use Option instead

Passing around object references that might be null is usually a NullPointerException waiting to happen. When references can be null, any piece of code using them need to check them. Unfortunately, failing to null-check references will not cause compiling errors and NullPointerExceptions can then happen at runtime. We all know this, yet we’re all guilty of doing it when coding in Java. As a matter of fact, null is only allowed in Scala in order to communicate with Java.

As an alternative, Scala provides the Option class. The idea is that any variable that might not be defined should be wrapped in an Option object. An object of class Option[A] will be in two possible states: either it is empty, or it is defined and contains an object of type A (not null).

Example: Repository lookup

Bad: Returning null if no result

object ProductRepository {  
    def findById(id: Int): Product = { … }

val product = ProductRepository.findById(123)  

The ProductRepository might not always return a result. If no matching Product is found, the temptation is to return null. However, this puts an implicit responsibility on the code using ProductRepository to check that there was indeed a result. Failing to check this will not prevent compilation and will potentially cause runtime exceptions later on.

Better: Returning Option

object ProductRepository {  
    def findById(id: Int): Option[Product] = { … }

val productOption = ProductRepository.findById(123)  
val productName = if (productOption.isDefined) else “???”  

The signature of findById now makes it clear that a result is not guaranteed, pushing the calling code to check before using it. However, using Option.get is a bad idea because the compiler will not ensure that Option.isDefined was checked beforehand.

Better: Using Option.fold (or pattern matching)

val productName = productOption.fold(“???”)(  

Option.fold covers both cases in one statement, removing the risk of forgetting to check for empty values.

This statement will set productName to "???" if productOption is empty, otherwise it will get the name of the product. If you are more comfortable with pattern matching notation, the following would be equivalent:

val productName = productOption match {  
    Some(product) =>
    None => “???”

Most scala collections functions will return Option objects when there might be no result (e.g. Map.get).

Using WartRemover to keep your Scala in line

Guidelines are a good start, unfortunately we all know that as a project progresses there can be multiple reasons (conscious or not) that they just end up forgotten somewhere on a Wiki page.

The Scala WartRemover tool has been a blessing for us as we started our journey with Scala. Rather than us trying to remember these guidelines while coding or reviewing Pull Requests, WartRemover verifies these rules at compilation time.

The earlier points are all parts of predefined Warts (i.e. things not to do) that will cause compilation to fail:

  • the use of var is forbidden, to disallow use of mutable variables
  • the use of collections from scala.collection.mutable is forbidden
  • the use of null is forbidden
  • the use of Option.get is forbidden

There are many more rules that will help your project stay clean, such as :
preventing early block termination (return and throw are disallowed)
enforcing static typing (isInstanceOf and asInstanceOf are disallowed)
keeping the code functional (methods with default arguments are disallowed)

Final thoughts...

Starting some real work on Scala after years of Java was a bit more challenging than expected, especially trying to get a real feel for functional-style programming (instead of trying to code Java in Scala!).

Adding WartRemover to our Scala projects forced us to refactor our code quite a bit and made us learn the correct way to code with functional programming principles in mind. If you are starting Scala or if you want to sanity check your existing projects, you should definitely give it a go!