If you’ve used Scala Future in a real application you probably realised that having a failed Future wrapping any Throwable isn’t the most efficient way to handle errors in your application.
I used Scala Future in this example but it applies equally to cats effects. Any type like IO[A] suffers the same issue in that regard as it doesn’t provide an error channel other than Throwable.
It’s basically the same issue as the undocumented Java RuntimeException. The only way to know what exceptions are thrown is to look at the implementation (when it’s possible).
Take the following function:
def doSomething(): Future[Unit]
From the return type you can guess it performs some side-effects but what happens if it fails? You get back a failed Future but what exception is wrapped in the Future?
There is no way to know apart from inspecting the implementation code.
As an alternative you’ve likely explored using Future[Either[Error, Result]] as a way to expose the error type in the signature. That’s much better but you’ve probably discovered that it makes all your for-comprehension structures much more tedious to write.
What used to be nice looking for-comprehension like this:
def getUser(email: String): Future[User] = ???
def getOrdersForUser(id: String): Future[List[Order]] = ???
// all items bought by a user
for {
user <- getUser(email)
orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)
Now becomes
def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[Either[Error, List[Order]]] = ???
for {
user <- getUser(email)
orders <- user.traverse(u => getOrdersForUser(u.id))
} yield orders.map(_.flatMap(_.items))
Fortunately there is a monad-transformer EitherT that solves the issue and makes your for-comprehensions look nice again.
def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[Either[Error, List[Order]]] = ???
(
for {
user <- EitherT(getUser(email))
orders <- EitherT(getOrdersForUser(user.id))
} yield orders.flatMap(_.items)
).value
That’s right but suddenly writing code has become a lot more painful: all the compilation errors become much more cryptic, highlighting the whole for-comprehension when you think everything’s correct.
Why is it so hard?
To make it short because Cats’ EitherT is too generic. It must work with any type F (not just Future or IO) and as a consequence it can’t make any assumptions on the properties of type F.
However when working with Future (or IO) we know that it is covariant. It’s defined as Future[+A] which means that a Future[A] is also a Future[B] if A is a subtype of B.
And Either is covariant as well so similarly a Future[Either[E, A]] is also a Future[Either[E, B]] if A is a subtype of B.
Unfortunately this is not the case with EitherT. EitherT[Future, E, A] is never a subtype of EitherT[Future, E, B] even when A is a subtype of B.
And that’s where all the compilation errors come from. The lack of covariance prevents the compiler to infer the types correctly and that’s why you always need to add type annotations when working with EitherT.
Take the following example
def getUser(email: String): Future[Either[Error, User]] = ???
def getOrdersForUser(id: String): Future[List[Order]] = ???
for {
user <- EitherT(getUser(email))
orders <- EitherT.right(getOrdersForUser(user.id))
} yield orders.flatMap(_.items)
And this doesn’t compile. why?
type mismatch;
found : UserId => cats.data.EitherT[scala.concurrent.Future,Nothing,List[Order]]
required: UserId => cats.data.EitherT[scala.concurrent.Future,Error,List[Ordered]]
Here we are again: a EitherT[Future, Nothing, A] is not a subtype of EitherT[Future, Error, A], although Nothing is a subtype of Error.
The fix is easy: just add a type annotation in the right place:
for {
user <- EitherT(getUser(email))
orders <- EitherT.right[Error](getOrdersForUser(user.id))
} yield orders.flatMap(_.items)
EitherT.right[Error] tells the compiler that the error type (which can’t be inferred from the method signature) is an Error (and not Nothing).
Having to fix all these kind of compilation error is annoying (and sometime quite time consuming, especially when people are not familiar with EitherT).
So the question is “Can we do any better?”
And of course we can! But first let’s have a look at what exactly is EitherT.
final case class EitherT[F[_], A, B](value: F[Either[A, B]])
Yes that’s right EitherT is just a simple case class whose only field is a F[Either[A, B]] so in our case a Future[Either[A, B]].
It also provides method like map and flatMap that operate on the Either inside the F.
The companion object provides convenience methods to create an EitherT from anything that’s not a F[Either[A, B]] (e.g. an A or a B or an Either[A, B] or a Future[B] or even an Option[B]).
As we know we’re working with Future (if you’re working with IO just replace Future with IO) let’s create our own type to wrap a Future[Either[E, A]].
final case class F[+E, +A](value: Future[Either[E, A]])
E emphasises that a Left holds an error case while a Right is for a success case.
Also note that F is covariant thanks to the +E, +A type parameters.
Then we need to implement all the method we’re going to use such as map, flatMap, …
def map[B](f: A => B)(implicit ec: ExecutionContext): F[E, B] = F(value.map {
case Right(a) => Right(f(a))
case Left(e) => Left(e)
})
def flatMap[EE >: E, B](f: A => F[EE, B])(implicit ec: ExecutionContext): F[EE, B] =
F(value.flatMap {
case Right(a) => f(a).value.map {
case Left(e) => Left(e)
case Right(a) => Right(a)
}
case Left(e) => Future.successful(Left(e))
})
Note that flatMap returns a F[EE, B] where EE is a super type of E. Why? to be able to keep the E of the initial F.
For example, if you EE and E are the same time you just end up with a F[E, B]. Now if they are different you end up with a F[E or EE, B] and because EE is a super type of E (E is also an EE) then you just have a F[EE, B]. That’s important for the type inference.
Having a map and a flatMap method allows you to write nice for-comprehensions again:
def getUser(email: String): F[Error, User]] = ???
def getOrdersForUser(id: String): F[Error, List[Order]] = ???
for {
user <- getUser(email)
orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)
You can now add a few convenience methods to create an F:
def success[A](a: A): F[Nothing, A] = F(Future.successful(Right(a)))
def fail[E](e: E): F[E, Nothing] = F(Future.successful(Left(e)))
def fromEither[E, A](v: Either[E, A]): F[E, A] = F(Future.successful(v))
def fromTry[A](ta: Try[A]): F[Throwable, A] = F(Future.successful(ta.toEither))
def fromFuture[A](fa: Future[A])(implicit ec: ExecutionContext): F[Throwable, A] =
F(fa.transform {
case Success(a) => Success(Right(a))
case Failure(e) => Success(Left(e))
})
As you can see F.success returns a F[Nothing, A] meaning that it can not fail.
And similarly F.fail returns a F[E, Nothing] meaning that it can not succeed.
F.fromTry and F.fromFuture are interesting because as they are built with a value that can fail with a Throwable they return a F[Throwable, A].
That’s really nice to be able to tell what the errors might be just by looking at the return type.
It also reminds you that when you build an F from a Future you need to think how you want to handle the Throwable.
It can be as simple as recovering into a default value:
def getUser(email: String): Future[User] = ???
val user: F[Nothing, Option[User]] = F.fromFuture(getUser(email)).handleError { e =>
logger.error(s"Failed to fetch user $email", e)
None
}
Now because with handled the error our user has a type of F[Nothing, Option[User]] making it clear that it won’t fail.
You may have a user or not (because of the Option type) but you won’t have an error.
What’s even nicer is that unlike cats’ EitherT all these types compose nicely. So to get back to our example you can now write:
def getUser(email: String): F[Error, User]] = ???
def getOrdersForUser(id: String): F[Nothing, List[Order]] = ???
for {
user <- getUser(email)
orders <- getOrdersForUser(user.id)
} yield orders.flatMap(_.items)
And there’s no more need to add type annotations to guide the compiler. The type inference works like a charm.
Not bad! but there’s still something missing: a better cats integration.
Remember how cats allows you to write things like:
val emails: List[String] = ???
val users: F[Error, List[User]] = emails.traverse(getUser)
Now that getUser returns an F it’s no longer possible. Why? because cats implicits requires an instance of Applicative[F] to be available in scope.
So let’s just add that. In fact we’re not going to add an instance for Applicative[F] but an instance of MonadError[F, E] because this is what our type F really is.
implicit def monadError[E](implicit ec: ExecutionContext): MonadError[({type M[A] = F[E, A]})#M, E] =
new MonadError[({type M[A] = F[E, A]})#M, E] with StackSafeMonad[({type M[A] = F[E, A]})#M] {
def pure[A](a: A): F[E, A] = F.success(a)
def handleErrorWith[A](fa: F[E, A])(f: E => F[E, A]): F[E, A] =
fa.handleErrorWith(f)
def raiseError[A](e: E): F[E, A] = F.fail(e)
def flatMap[A, B](fa: F[E, A])(f: A => F[E, B]): F[E,B] = fa.flatMap(f)
}
this creates a MonadError[F[E, ?], E] for any F. The scary ({type M[A] = F[E, A]})#M is just a type lambda. It creates a single parameter type M[A]that applies the A in F[E, A] and keeps the E constant.
It can be written simply as F[E, ?]using the kind-projector plugin.
We also used the StackSafeMonad because Future is stack-safe so it is safe to do so.
Et voilà! With a few lines of code (about 150 lines in the gist) you have a way to write nicer programs without the struggles of using cats EitherT.
Of course there are better solutions available. E.g. ZIO solves this problem elegantly and provides better abstraction and performance than simply wrapping a Scala Future into a case class.