Finding Issues in Scala With Wart Remover

Update, Sep. 15, 2014 16:09 UTC: Much wordsmithing.

Update, same day: forgot to include how to run the thing. Now added!

Some recent work I’ve been doing led me to investigate the use of Wart Remover to catch bugs.

I’ve discovered that Scala has quite a few insiduous corner cases. For one, the combination of sub-typing with type inference leads to dangerous inferred types from time to time. Further, Scala provides various type-safety escape hatches that lead to reduced effectiveness of static analysis.

In this post, I’ll take a look at how Wart Remover addresses these issues, and why addressing all issues found by it can lead only to strictly better code.

What is Scala’s Wart Remover?

Wart Remover is a linter that is aware of the escape hatches and inference corner cases in Scala’s type system. It works with the type system to improve the reliability of your code. It catches the following issues:

I’ll look into each of these in more detail in the following sections.

Setting Up Wart Remover

Wart Remover is pretty easy to set up for SBT-based build systems. I’ll cover the steps below, including a current corner case that the docs don’t cover. Three files are needed:

sbt.version=0.13.5
resolvers += Resolver.sonatypeRepo("releases")

addSbtPlugin("org.brianmckenna" % "sbt-wartremover" % "0.11")
scalaVersion := "2.11.2"

addCompilerPlugin("org.brianmckenna" %% "wartremover" % "0.10")

wartremoverWarnings ++= Warts.allBut(Wart.NoNeedForMonad)

scalacOptions ++= Seq("-deprecation", "-Xlint")
wartremoverWarnings ++= Warts.allBut(Wart.NoNeedForMonad)

This line above in build.sbt is needed because NoNeedForMonad is currently broken. It would normally warn about where Applicative-dependent code would suffice.

As to why you might prefer Applicative implementations over Monadic ones, I’ll quote a paper referenced in the footnotes:

The moral is this: if you’ve got an Applicative functor, that’s good; if you’ve also got a Monad, that’s even better!

…the dual of the moral is this: if you want a Monad, that’s good; if you only want an Applicative functor, that’s even better!

Applicative computations are composable, are more amenable to analysis, and are frequently more succinct than Monadic computations. The short of it is: if you don’t need dynamic decision making in your computation, e.g., (if pred then path1 else path2), prefer Applicatives to Monads. 1 2

Running Wart Remover

Given the set up above, running the thing is as simple as issuing a compile:

$ sbt compile
...
[warn] /.../dev/proj/src/main/scala/client/HttpPlan.scala:19: Inferred type containing Nothing
...
[warn] 159 warnings found
[success] Total time: 141 s, completed Sep 15, 2014 1:01:45 AM

Issues: Type Inference Failures

Scala’s type system, for better or worse, features subtyping available throughout. In more common terms, it means that inheritance is available as a tool for extending types. Consequentially, it is impossible for the type inference engine to arrive at safe conclusions in some cases. This matter is worsened by a design decision that let’s Scala infer Any as a valid type for mixed constructs. Any is much akin to C’s (void *) - anything goes. Information is lost, and the confidence we can have in the compiler’s conclusions is reduced significantly.

Thankfully, Wart Remover catches these sorts of cases. If you see any of the warnings below, the fix is usually as simple as providing type annotations. In the case of Nothing type inferred, this literally means your program will crash here if your code is left as-is.

Note: the let it crash (pg. 107) approach doesn’t work in Scala because of a lack of built-in process supervision. This is not Erlang. Do not take crashing paths for granted.

Any Type Inferred

object AnyIssue {
  val any = List(1, true, "three")
}

Nothing Type Inferred

object CrashForNothing {
  val crash = ???
  val wrongType = List.empty
}

Product Type Inferred

object ProductOfSubtyping {
  val x = List((1, 2), (1, 2, 3))
}

Serializable Type Inferred

object ProductOfSubtyping {
  val x = List((1, 2), (1, 2, 3))
}

Issues: Partiality and Type Evasion

Partiality means that your functions don’t cover all values that could be passed to it. List functions are a good example of this. The ListPartials below all crash when an empty list value is encountered.

The solution to partiality is often simple: use a function that returns an Option or an Either for values you’re not handling. If you want to be more explicit, a sum-type via case classes could be even better, say, for a Weekday data type.

Type evasion is my own terminology. I refer to the use of back doors to evade the benefits of a type system. An example of this would be to use unsafePerformIO in Haskell land, and should be treated with the same level of suspicion. Whenever Wart Remover warns you of these sorts of type system evasions, heed the warning as soon as possible.

Partial Methods on Lists

import scala.collection.immutable.List._

object ListPartials {
  val crash1 = List().head
  val crash2 = List().tail
  val crash3 = List().last
}

Get Used to Bypass Type-Safety

object BadGets {
  val x = List().headOption.get
}

Issues: Runtime Type Coercion

Run-time type coercion is occasionally a valid technique. Much research3 has gone into type-safe type coercion. asInstanceOf and isInstanceOf are not the fruits of that research and should be considered suspect. Think of it as trying to force a Circle shape into a Triangle mold - you’ll have lost bits of the Circle and might not even have a full Triangle!

IsInstanceOf: Runtime Type Detection

object IsInstance {
  val x = "1".isInstanceOf[String]
}

AsInstanceOf: Unsafe Type Coercion

object AsInstanceOf {
  val inst = 1.asInstanceOf[String] + "1"
}

Any String Add Conversion

object AnyStringIssue {
  val any = {} + "cat"
}

Issues: Miscellaneous Obstacles

The remaining classes of warnings that Wart Remover provides are more subtle. Consider them as design flaws that could lead to maintenance troubles in the future. This is particularly true of the use of var.

Use of Default Arguments

object DefArgs {
  def connect(secure: Boolean = false): Unit = {
    println(secure)
  }
}

Here are some more thoughts on why this is a questionable practice.

Statements Should Return Unit

object TroublingStatements {
  def x() = {
    10 // returns Int, indicating this line does no useful work
    println("non-unit above")
  }
}

Null Value Used

object NPEWaitingToHappen {
  val nulled = null
}

Return Used

object ReturnFromWhenceYouCame {
  def plus(x: Int, y: Int):Int  = {return x + y}
}

Var Used

object RejectEquationalReasoning {
  var x = 100
}

Why Fixing These Issues Can’t Hurt

Unlike linters from languages like C, C++, and Java, Wart Remover does not give false positives. This means that if a particular construct is flagged by Wart Remover, you should think very carefully before choosing to ignore it. To understand why, read on. It all comes down to the promise of type safety within the realm of a sound type system.

Let’s look at the definition of type safety as given by Benjamin Pierce in Types and Programming Languages:

Safety = progress + preservation 4 5

Progress

If t is a well-typed term (that is, t : T for some T), then either t is a value or else there is some t’ with t -> t’.

Preservation

If t : T and t -> t’ then t’ : T

Progress and Preservation together mean that we can count on sections of code to behave as we wrote them.

Let’s look at each class of issues caught by Wart Remover in turn and reason why fixing them can’t make your code any worse.

Type Inference Failures

In the best case, all we need to do is add a type signature to aid the compiler. Type signatures do not affect the run-time behavior of code. Therefore, this cannot hurt.

In the worst case, we’re changing something that would’ve crashed to something else (Nothing inferred). Consider that the worst thing that can happen to your system is to crash. As a result, anything we do to address this warning cannot leave us in worse place.

Partiality

If you are addressing partial List methods, for example, the solution would typically be to use a method that returns an Option. What you’ve done is removed the possibility of crashing at the cost of one more line of code.

In the shared common case, upon receiving a non-empty list value, the code behaves as before. Upon receiving an empty list value, the old code crashes and the new code does something. At least in the new code, that something is predictable and under your control.

The use of get introduces partiality and I reason about it as above.

Coercion

Here’s an example of coercion in action:

scala> 1.asInstanceOf[String]
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String

Note: this is a run-time error. Contrast this to a compile-time error:

scala> "1" * "a"
<console>:8: error: type mismatch;
 found   : String("a")
 required: Int
              "1" * "a"
                    ^

It is a toy example, but what’s key here is that coercion leads to run-time errors. This means that it is not safe, in the sense defined earlier. Given that coercion in Scala is not guaranteed to be safe, consider the following -

When you employ coercion, you are choosing to take the burden of what a stretch of code does into your own (and your team’s) heads, rather than trusting the compiler. Case analysis is difficult here. Resolving the issue means you’ve found a way to encode what needs to be encoded in the type system. Choosing to ignore the issue means an implementation detail must now be kept in mind by those who will maintain your system. You may pass it off to a unit test, but now you have to maintain that test suite, as well.

Remaining Issues

These are more subtle to reason about and fix. I’ll pass on discussing why the use of null, var, return, and default arguments can lead to maintenance problems.

Closing Thoughts

I’ve written a lot at this point. I stewed over the subject for a few weeks. Most of what I’ve written in this post comes down to trusting a type system to prove that what you’ve implemented is consistent.

With Scala, you have to work with the quirks of the language to be able to trust the type system. It’s capable enough. It’s on par with Haskell, in many respects. Wart Remover knows about those quirks.

I wrote this post because I want to advocate for safer Scala. We have the tools. Let’s use them!

  1. You can read this SO answer for a succinct explanation.↩

  2. This functional pearl explains the Applicative/Monad difference in detail.↩

  3. Type-safe, cost-free type coercion has been the subject of recent research in Haskell land. See Safe Zero-cost Coercions for Haskell↩

  4. Types and Programming Languages (TaPL), my favorite book for understanding the basics of type systems.↩

  5. In the absence of TaPL, I turn to Michael Bernstein’s post for a succinct summary of type safety.↩