The Developer’s Guide to Scala Implicit Values (Part II)

Imagine a rectangular grid of cells, in which each cell has a value – Either black (dead) or white (alive). And imagine that:

  1. Any live cell with two or three live neighbors survives for the next generation.
  2. Any cell with four or more neighbors dies from overpopulation.
  3. Any cell with one or no neighbors dies from isolation.
  4. Any dead cell with exactly three neighbors comes to life.

 

These are the four simple rules of Conway’s Game of Life . You could hardly imagine a simpler set of rules to code on your computer and you wouldn’t expect any interesting result at all, but…

Behold the wonders of its hidden might!

The Conway’s Game of Life is an example of how a simple set of rules can produce amazing and totally unexpected results.

In the previous post, we agreed that the rules governing Scala’s implicit context weren’t complex at all, all of it can be summed up as: Mark some parameters as special, let the compiler know there are some values and some functions it can use when it is struggling with your code compilation… and hope it can manage to fill the gaps.

No matter how simple the implicit context rules are, they are definitely  more complex than Conway’s. And, as in his little game of life, they bring complex results that, sometimes, make Scala newcomers sweat. However, under the developer’s control, they can become the glue that holds the most important Scala patterns together – the dark matter behind its visible universe. In this article we’ll walk through some examples of the marvels implicits can lead to.

Scala’s Dark Matter

Whereas it is theorised that some non-visible matter is responsible for helping maintain the galaxies together, we know for sure that some non-straightforwardly-visible values and functions hold many Scala constructions together. Let’s see some of these constructions.

Implicits application 101: The hidden context

Imagine you’re enjoying a session of peer programming. Your mate has the keyboard while you keep your eyes wide open, eager to suggest changes and solutions. Wouldn’t you find it stupid to repeat your friend’s name every time you address her/him? It is implicit that you are talking to her/him and not to another person who may be walking around!

Why should it be any different in a block of code which repeatedly interacts with a service?!

If you come from Java you might be thinking: “What are you, dude, some kind of slacker?”. Well, you also keep writing “;”s which, let’s be honest, are ignored by everybody’s brain…  Lets give a concrete example from Oracle’s documentation on Java Futures:

It’s like saying: “Hey you! Executor, do this“, “Hey you! Executor, do this other thing”. It would be more comfortable, by far, to just agree to use a concrete executor at the beginning of a code block, thus avoiding writing its identifier over and over again. Take a look at fairly equivalent code written in Scala:

Now take a closer (unfolded) look at the creation of the first future:  val futureA = Future[String] { ... } . This is calling for Future objects to apply factory method…

…with the following method signature indicating it can get its executor parameter from an implicit value:

Sound familiar? apply’s body parameter is to Java’s Executor#execute command parameter what the executor implicit parameter is to the Executor instance repeatedly and explicitly referred to in Java.

This pattern easily comes up when developing  a context dependent library and it isn’t difficult at all to develop a quick example.

You are developing a platform game in which your open source hero, Tux, moves around a stage, dodging and killing enemies, collecting rewards…

Your task is to write a comfortable collision detection module whereby its users can easily determine how the game reacts to different positions of our hero, at concrete times given a scenario. A first approach could be something like…

and it could be used to run the game loop step by step as follows …

Isn’t stage a context in the game solution? It doesn’t vary over a game-loop step (runGameFrame) and is constantly and explicitly used by many of the operations performed. By noticing this and making it an implicit parameter for these operations we get a much clearer and concise code:

Types, tourists at run-time

Well, gluing contexts to methods, functions or constructor calls is cool, saves time and keystrokes and those are great advantages of a language for which developers aren’t paid by the written byte.

Now is time to meet a pattern providing something you wouldn’t get by other means: Run-time types awareness.

Types are citizens of the compilation time, they determine how – or if – your program will compile. However, without the help of the goddess of reflection, the only way to make decisions upon value types at run-time is through pattern matching:

However, the underlying mechanism behind type matching is nothing else than JVM’s  instanceof operator which is constrained by JVM’s type erasure. This means that all types generated by a type generator (a genetic type) are classified as instances of the same class. Have you ever tried to do something like the following?

If you type the above code into Scala’s REPL it’ll yield: “It’s a list of integers” Ups!

The matter is that the above code is, in some way, like writing the following Java snippet where the parametric type has completely disappeared from the list type:

As the first condition is true, we can understand why the pattern match isn’t working. 

What a shame of a language if we are not able to make such a decision at run-time! … But don’t run from Scala just yet! It provides a way. It always does…

The way to do it, is by leveraging our dark matter: Implicit values. The Scala compiler automatically provides implicit values for any of the types generated by  TypeTag[T] . If you don’t believe me, check for yourself:

The point now is to use these labels to make decisions based on the types at run time. Remember the unsatisfactory attempt with pattern matching? Let’s try again. This time using the new cutting edge dark matter drive:

Yeah! OK, so far so good – but don’t think this is the only way, or even the best way to defeat type erasure. But this post is about implicit things.

I’d be more than happy to show you more about types at run time, but this is like a Star Wars film (the oldies) – if you want to know how the Death Star plans were stolen – chapter and verse – you’ll have to wait for Rogue One

However, there is something related to implicits and to implicits only, which make the above code a lot less verbose…

Implicit type bounds

Coming back to  whatIsIt[T](l: T)(implicit tag: TypeTag[T]): Unit . Look at the second parameter list. Yes, the one for implicit parameters. It is worth noting that the code pattern for methods like  someMethod[T](...)(implicit SomeParametricType[T]) is so common that the language provides a shortcut for it:  someMethod(...)[T : SomeParametricType] and our  whatIsIt could make good use of it:

Don’t you love this language?!

To conclude this interlude, a last piece of advice: Don’t think that implicit type bounds are incompatible with other types of bounds. The following piece of code is not only correct, but also represents a common use of type bounds:

By the way, the implicit parameter for the TypeTag is known as the implicit evidence for T.

The magnetic attraction of the “classy type”

Have you ever tried to add functionality to a case class? Have you ever wanted to work with a third party class as if it was a subclass of one of yours?

Farm Management Ltd. has given us a great library with classes to represent animals in a farm management system. Among others, it contains the classes: Cat, Cow, Dog and Unicorn. All these classes implement the interface Animal…

Animal establishes that all its implementations should offer the operation  takeCareOf(caretaker: PersonId, date: Date): ScheduledActivity  which returns a ScheduledActivity value representing a programmed activity in the managed farm operations.
Our goal is to expand the farm management system so it can also manage activities related to equipment maintenance – such as fencing, fixing machinery, … we then provide classes representing “things to be maintained” which are not animals:

Equipment establishes the operation  fix(worker: PersonId, date: Date): ScheduledActivity . So far so good… but now a problem arises: How will our management system be able to integrate animals to be taken care of with machines to be fixed in a common scheduler for all the farm activities? Remember, we CAN NOT modify the class Animal, that is: We CAN NOT create a common ancestor for  Equipment and  Animal to polimorphically work with their implementations’ instances.

An experienced Java developer would probably say something like “What’s the matter bro?!  Wrap it up and carry on!” meaning: Just create a wrapper for the class provided by Farm Management Ltd. 

This is great, but not so comfortable. All Animal instances need to be manually boxed within AnimalWrapper instances. That is, we need a kind of force to glue the farm element families together. e.g:

In the above example, that force is the explicit boxing of animals ( AnimalWrapper(animal) ). What an inconvenience to manually box every single Animal instance you want to integrate in your management system!

No! We are well into the 21st century. Besides, we agree that we have been kind of slack, don’t we? There must be a better way!

What we are actually searching for is a way to categorise types, or classes, after their declaration, some kind of delayed type classification. By using old school OOP inheritance we are classifying our types when they are defined, therefore the integration problems with third party entities. To overcome this kind of situation, it would be cool to have a mechanism which allows us to set classes of entities with a defined set of operations and to link types of instances to some classes whenever needed, not only at definition time. This idea is not new and such a mechanism exists for Haskell since 1990: Type Classes.

This is the way you could solve the problem in Haskell – Don’t run away just yet! I promise it will be quick and painless:

You don’t need to understand the above code beyond the idea of having a way to say that some types can be associated with a certain interface even after definition. Remember:

Type classes are classes of types (classes of classes in Scala), not classes of instances

Despite Scala’s type system being closer to Java’s than to Haskell’s, we can use our magical dark matter – yes, our implicit values and conversions – to build Type Classes.

  • Step 1, Untangling the classes hierarchy: The above classes graph is simplified since we will break the hard-coded composition relation with Animal interface.

  • Step 2, Declaring the type class (Maintainance) and its interface as in

    Which, in Scala, is a trait:
  • Step 3, Making classes part of the type class: In a delayed fashion of course…
  • Step 4, Making use of type class interface throught the implicit context:

And BOOM! There it is, a kind of Haskell type system embedded in Scala and you know… classes – of types – have always existed in Haskell.

However, there is something not so cool about these kind of type classes in which you need to pass the instance to the operations. It is possible to push the limits of OOP into this realm thus enabling the lovely infix notation we always feel comfortable with. Thus applying what has been called …

 

which was first described on Spray’s development blog.

The latter article is so well written and so clearly explains the concept, that copying its ideas would be not only useless, but most probably misleading.

There is little differences between the magnet pattern and the type classes pattern as described above: Both rely on implicit values to link types to their class, that is to a set of operations. There is, however, a main difference which clearly sets both patterns apart:

  • Type classes rely on implicit objects, that is: values of the type TypeClassClass[T]   – Where T is the type to be associated to the type class – thus enabling the type class of frozen operations through an implicit evidence within the implicit context.Another example of type class is Scala is the  Ordering[T] trait:
  • With magnets, on the other hand, the operations are not fixed as they depend on value parametrization which leads to an object containing the implementation of the operation for that parametrization. This implies that magnets are not provided by implicit values accessed through implicit evidence but, otherwise, are implicit conversions from the operation parameters to the implementation. e.g:

    An example of use:

It is possible to fiddle with Haskell-like type classes and Magnet patterns in order to explore a vastly wide space of variations between them.

Other notorious Implicit Dimension citizens

This article is already “TLDR” so let’s briefly visit a couple of interesting implicit value/conversion cases.

Implicit classes & Implicit object

When we were getting to know the Type Classes nuts and bolts we saw that a singleton object can be tagged as implicit:

It seems pretty obvious that it is just a regular singleton object tagged as implicit and, therefore, becomes also an implicit value to be used as an actual value of any call to a function with an implicit formal parameter of type T :

The reasoning applied to understand what implicit objects are, can be extended to implicit classes. Whereas singleton object constructors do not require any parameter and can be built straightaway – without providing any parameter. There is a special type of class, the type of classes whose constructor receives exactly one parameter, than can be regarded as an implicit conversion from that parameter to instances of themselves (of the classes):

Convenient to write DSL, isn’t it?

Dangers and pitfalls

A regular matter universe is full of perils, it seems reasonable to think that playing with the mysteries of dark matter will bring problems of its own…

Multiple completions

When I read the words “error: ambiguous implicit values” I feel like Han Solo entering an asteroid field when trying to escape from the empire…

But there is not so much to fear. To solve this problem, at least in a quick and dirty way, is a piece of cake. It just means that a method is implicitly expecting a parameter of type, let’s say, T and the implicit context has more than one value of that type to fill the gap which leads to ambiguity… The compiler is politely telling us: Dear developer, please, can you be a little more specific?

The easiest solution to this problem is to explicitly provide an actual parameter of that type to the function call:

But we are not despicable bounty bug hunters, are we? As Scala developers & software engineers we should strive for clean solutions and to find the root of the problem. In this case, as in most similar cases, the implicit context gets populated with implicit values of the same type from different imports. Just removing one of them would be the wiser approach.

By the way, tools and IDEs – such us IntelliJ (with its Scala Plugin) – can help locate were the implicit values come from. In the case of the mentioned IDE, selecting the calling expression, the one receiving ambiguous implicit actual parameters, and using the key shortcut CONTROL+SHIFT+P will show a floating window showing the part of the implicit context to be used by the expression and the source of its values.

It may happen that we can’t get rid of one of the imports because it may contain other useful implicit values, in such cases, the problematic values can be avoided by diverting their symbol import:

Recursive completions: What can be cooler than a run-time recursion Stack Overflow Error?

A recursion stack overflow error which you can’t see, of course!

To wrap this post up, let’s take a look at my favorite error in the Scala ecosystem. We have all come across the run-time stack overflow errors when defining wrong stop conditions – or none at all –  def soFarSoGood: Long = 1 + soFarSoGood .

Well, there is a misuse of implicit completions which can lead to unexpected crashes under similar circumstances: Implicit auto-completion.

Be aware of cross implicit conversions and…

Enjoy

2 Comments

  1. Bob Gleasonsays:

    In the Implicit Type Bounds section, I think you want:
    def whatIsIt[T : TypeTag](l: T): Unit = {
    if(typeOf[T] =:= typeOf[List[Int]]) println(“It’s a list of integers”)
    else if(typeOf[T] =:= typeOf[List[String]]) println(“It’s a list of strings”)
    }

    There is no “tag” value any more in your original, so it doesn’t compile as written.

    • You’re right Bob,

      There is a missing line making the tag explicit through implicitly: It has been added to the text:

      val tag = implicitly[TypeTag[T]]

      You might be thinking, “OK, using type bounds doesn’t help much if I need to declare the tag identifier and use implicitly in order to access de that”. And you wouldn’t be wrong in the context of the example.

      Implicit type bounds are useful for interfaces or method definitions where the implicit value ins’t being explicitly but propagated through calls to methods expecting to receive the same value as an actual implicit parameter. e.g:

      trait Action
      object ManageListOfIntegers extends Action with … { … }
      object ManageListOfStrings extends Action with … { … }
      object NoOp extends Action with … { … }

      def determineCallback[T](implicit tag: TypeTag[T]): Action = {
      if(typeOf[T] =:= typeOf[List[Int]]) ManageListOfIntegers
      else if(typeOf[T] =:= typeOf[List[String]]) ManageListOfStrings
      else NoOp
      }

      def serveRequest[T : TypeTag](rq: Request[T]): Try[Result] = {


      val action = determineCallback

      }

Leave a comment

Please be polite. We appreciate that. Your email address will not be published and required fields are marked