Categories
Programming

The problem with EitherT

If you’ve used Scala Future in a real application you probably realised that having a failed Future wrapping any Throwable isn’t the most efficient way to handle errors in your application.

I used Scala Future in this example but it applies equally to cats effects. Any type like IO[A] suffers the same issue in that regard as it doesn’t provide an error channel other than Throwable.

It’s basically the same issue as the undocumented Java RuntimeException. The only way to know what exceptions are thrown is to look at the implementation (when it’s possible).

Take the following function:

def doSomething(): Future[Unit]

From the return type you can guess it performs some side-effects but what happens if it fails? You get back a failed Future but what exception is wrapped in the Future?

There is no way to know apart from inspecting the implementation code.

As an alternative you’ve likely explored using Future[Either[Error, Result]] as a way to expose the error type in the signature. That’s much better but you’ve probably discovered that it makes all your for-comprehension structures much more tedious to write.

What used to be nice looking for-comprehension like this:

def getUser(email: String): Future[User] = ???
def getOrdersForUser(id: String): Future[List[Order]] = ???

// all items bought by a user
for {
 user <- getUser(email)
 orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)

Now becomes

def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[Either[Error, List[Order]]] = ???

for {
  user <- getUser(email)
  orders <- user.traverse(u => getOrdersForUser(u.id))
} yield orders.map(_.flatMap(_.items))

Fortunately there is a monad-transformer EitherT that solves the issue and makes your for-comprehensions look nice again.

def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[Either[Error, List[Order]]] = ???

(
  for {
    user <- EitherT(getUser(email))
    orders <- EitherT(getOrdersForUser(user.id))
  } yield orders.flatMap(_.items)
).value

That’s right but suddenly writing code has become a lot more painful: all the compilation errors become much more cryptic, highlighting the whole for-comprehension when you think everything’s correct.

Why is it so hard?

To make it short because Cats’ EitherT is too generic. It must work with any type F (not just Future or IO) and as a consequence it can’t make any assumptions on the properties of type F.

However when working with Future (or IO) we know that it is covariant. It’s defined as Future[+A] which means that a Future[A] is also a Future[B] if A is a subtype of B.

And Either is covariant as well so similarly a Future[Either[E, A]] is also a Future[Either[E, B]] if A is a subtype of B.

Unfortunately this is not the case with EitherT. EitherT[Future, E, A] is never a subtype of EitherT[Future, E, B] even when A is a subtype of B.

And that’s where all the compilation errors come from. The lack of covariance prevents the compiler to infer the types correctly and that’s why you always need to add type annotations when working with EitherT.

Take the following example

def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[List[Order]] = ???

for {
  user <- EitherT(getUser(email))
  orders <- EitherT.right(getOrdersForUser(user.id))
} yield orders.flatMap(_.items)

And this doesn’t compile. why?

type mismatch;
 found   : UserId => cats.data.EitherT[scala.concurrent.Future,Nothing,List[Order]]
 required: UserId => cats.data.EitherT[scala.concurrent.Future,Error,List[Ordered]]

Here we are again: a EitherT[Future, Nothing, A] is not a subtype of EitherT[Future, Error, A], although Nothing is a subtype of Error.

The fix is easy: just add a type annotation in the right place:

for {
  user <- EitherT(getUser(email))
  orders <- EitherT.right[Error](getOrdersForUser(user.id))
} yield orders.flatMap(_.items)

EitherT.right[Error] tells the compiler that the error type (which can’t be inferred from the method signature) is an Error (and not Nothing).

Having to fix all these kind of compilation error is annoying (and sometime quite time consuming, especially when people are not familiar with EitherT).

So the question is “Can we do any better?”

And of course we can! But first let’s have a look at what exactly is EitherT.

final case class EitherT[F[_], A, B](value: F[Either[A, B]])

Yes that’s right EitherT is just a simple case class whose only field is a F[Either[A, B]] so in our case a Future[Either[A, B]].

It also provides method like map and flatMap that operate on the Either inside the F.

The companion object provides convenience methods to create an EitherT from anything that’s not a F[Either[A, B]] (e.g. an A or a B or an Either[A, B] or a Future[B] or even an Option[B]).

As we know we’re working with Future (if you’re working with IO just replace Future with IO) let’s create our own type to wrap a Future[Either[E, A]].

final case class F[+E, +A](value: Future[Either[E, A]])

E emphasises that a Left holds an error case while a Right is for a success case.

Also note that F is covariant thanks to the +E, +A type parameters.

Then we need to implement all the method we’re going to use such as map, flatMap, …

def map[B](f: A => B)(implicit ec: ExecutionContext): F[E, B] = F(value.map {
  case Right(a) => Right(f(a))
  case Left(e) => Left(e)
})

def flatMap[EE >: E, B](f: A => F[EE, B])(implicit ec: ExecutionContext): F[EE, B] =
    F(value.flatMap {
      case Right(a) => f(a).value.map {
        case Left(e) => Left(e)
        case Right(a) => Right(a)
      }
      case Left(e) => Future.successful(Left(e))
    })

Note that flatMap returns a F[EE, B] where EE is a super type of E. Why? to be able to keep the E of the initial F.

For example, if you EE and E are the same time you just end up with a F[E, B]. Now if they are different you end up with a F[E or EE, B] and because EE is a super type of E (E is also an EE) then you just have a F[EE, B]. That’s important for the type inference.

Having a map and a flatMap method allows you to write nice for-comprehensions again:

def getUser(email: String): F[Error, User]] = ???
def getOrdersForUser(id: String): F[Error, List[Order]] = ???

for {
  user <- getUser(email)
  orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)

You can now add a few convenience methods to create an F:

def success[A](a: A): F[Nothing, A] = F(Future.successful(Right(a)))

def fail[E](e: E): F[E, Nothing] = F(Future.successful(Left(e)))

def fromEither[E, A](v: Either[E, A]): F[E, A] = F(Future.successful(v))

def fromTry[A](ta: Try[A]): F[Throwable, A] = F(Future.successful(ta.toEither))

def fromFuture[A](fa: Future[A])(implicit ec: ExecutionContext): F[Throwable, A] =
    F(fa.transform {
      case Success(a) => Success(Right(a))
      case Failure(e) => Success(Left(e))
    })

As you can see F.success returns a F[Nothing, A] meaning that it can not fail.

And similarly F.fail returns a F[E, Nothing] meaning that it can not succeed.

F.fromTry and F.fromFuture are interesting because as they are built with a value that can fail with a Throwable they return a F[Throwable, A].

That’s really nice to be able to tell what the errors might be just by looking at the return type.

It also reminds you that when you build an F from a Future you need to think how you want to handle the Throwable.

It can be as simple as recovering into a default value:

def getUser(email: String): Future[User] = ???

val user: F[Nothing, Option[User]] = F.fromFuture(getUser(email)).handleError { e =>
  logger.error(s"Failed to fetch user $email", e)
  None
}

Now because with handled the error our user has a type of F[Nothing, Option[User]] making it clear that it won’t fail.

You may have a user or not (because of the Option type) but you won’t have an error.

What’s even nicer is that unlike cats’ EitherT all these types compose nicely. So to get back to our example you can now write:

def getUser(email: String): F[Error, User]] = ???
def getOrdersForUser(id: String): F[Nothing, List[Order]] = ???

for {
  user <- getUser(email)
  orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)

And there’s no more need to add type annotations to guide the compiler. The type inference works like a charm.

Not bad! but there’s still something missing: a better cats integration.

Remember how cats allows you to write things like:

val emails: List[String] = ???
val users: F[Error, List[User]] = emails.traverse(getUser)

Now that getUser returns an F it’s no longer possible. Why? because cats implicits requires an instance of Applicative[F] to be available in scope.

So let’s just add that. In fact we’re not going to add an instance for Applicative[F] but an instance of MonadError[F, E] because this is what our type F really is.

implicit def monadError[E](implicit ec: ExecutionContext): MonadError[({type M[A] = F[E, A]})#M, E] =
    new MonadError[({type M[A] = F[E, A]})#M, E] with StackSafeMonad[({type M[A] = F[E, A]})#M] {
      def pure[A](a: A): F[E, A] = F.success(a)

      def handleErrorWith[A](fa: F[E, A])(f: E => F[E, A]): F[E, A] =
        fa.handleErrorWith(f)

      def raiseError[A](e: E): F[E, A] = F.fail(e)

      def flatMap[A, B](fa: F[E, A])(f: A => F[E, B]): F[E,B] = fa.flatMap(f)
}

this creates a MonadError[F[E, ?], E] for any F. The scary ({type M[A] = F[E, A]})#M is just a type lambda. It creates a single parameter type M[A]that applies the A in F[E, A] and keeps the E constant.

It can be written simply as F[E, ?]using the kind-projector plugin.

We also used the StackSafeMonad because Future is stack-safe so it is safe to do so.

Et voilà! With a few lines of code (about 150 lines in the gist) you have a way to write nicer programs without the struggles of using cats EitherT.

Of course there are better solutions available. E.g. ZIO solves this problem elegantly and provides better abstraction and performance than simply wrapping a Scala Future into a case class.

Categories
Programming

Streaming patterns with fs2

After an overview of cats-effect which deals with single effect it feels natural to have a look at fs2 and see how multiple effects can be combined into a stream.

As we’ve covered some common streaming patterns with Akka stream a while ago it’ll be interesting to see how they compare with each other.

Categories
Programming

Cats-effect, an overview

After many applications written using Scala’s Futures, Akka Actors or Monix,… Cats-effect is now my favourite stack to write Scala programs. Why is that?

Well, it makes your code easier to write and reason about while providing good performances.

Categories
Programming

Dealing with errors

As a programmer we don’t really like to deal with errors. We like to focus on the happy path – the one that provides value – and deal with the errors later because … well, we have to do it!

However dealing with failures is crucial if we don’t want our program to stop on the first error it encounters.

But still we don’t want to mix the “clean” code of the happy path with the “dirty” error handling code. And in fact this is what exceptions were suppose to bring: A clean happy path in a try statement and the error handling code in a catch statement. You know everything clean and separated.

In Scala we have a rich type-system which gives us more options to handle errors. But before we start let’s be clear by what we mean by errors.

Categories
Programming

Leveraging the type system to avoid mistakes

We all know we should write tests to make sure our system behaves as it is supposed to.

Surely tests are necessary to ensure correctness of our programs but they only depend on what the programmer is willing to test (or can think of testing).

What I mean is that there will always be gaps in the test coverage, like uncovered corner cases or improbable combinations of events, …

In Scala we have a powerful type system that we can use to help us avoid some mistakes.

Why would you bother writing a test to make sure a function handle some corner cases correctly when you can use the type system to make sure such cases won’t ever happen.

Categories
Programming

Refined types, what are they good for?

Type refinement is all about making the types more precise. But why would do that? Because using the correct types makes your program safer as you reduce the possibility to introduce bugs.

First let’s think about types. For instance String is a type we use all the time. A variable of type String can have many different values (in theory an infinity of values) but it’s quite unlikely that all these values make sense in our application.

Categories
Computing Programming

Akka stream interface for gRPC

Back from holidays let’s continue with some of my favourite topics: AkkaStreams and gRPC.

We’ve already seen how can take advantage of the ScalaPB code generation tool to generate new interfaces (GRPCMonix) on top of the grpc-java implementation or to create new tools to integrate gRPC with other services (GRPCGateway).

Similarly to GRPCMonix which provides a Monix interface – Task, Observable – on top of gRPC, it’s possible to develop an AkkaStream interface on top of gRPC.

Categories
Programming

Introduction to Tagless final

In this previous post we’ve seen that before using Scala’s Future it might be worth taking some time to think of the use cases (especially error cases), the execution model we need, … as it might be more advantageous to choose a solution like Monix’s Task (although not available in standard library) to gain finer control over the execution. However some might not be able to make a decision at this stage and like to keep their options open. Let’s see how we can revisit our product repository in such way that we don’t have to make a decision too early.

Categories
Programming

Scala Futures vs Monix Tasks

In this previous post we saw how Scala Futures work and why they need an implicit ExecutionContext to run their computation. While there is some trick to pass the ExecutionContext, it’s usually cumbersome and clutter the code.

So the question really is: Do you need an ExecutionContext everywhere? Well, you do as long as you use Futures, but is there any alternatives?

Categories
Computing Programming

Understanding Scala Futures and Execution Contexts

Viktor Klang recently published a set of useful tips on Scala Futures. While being widespread and heavily used, people (especially newcomers) are still experiencing problems working with Scala Futures.

In my opinion many of the problems come from a misunderstanding or misconception on how Futures work. (e.g. the strict nature of Futures and the way they interact with Execution Contexts to name a few).

That’s enough of an excuse to dive into the Scala Future and ExecutionContext implementation.