In this previous post we saw how Scala Futures work and why they need an implicit ExecutionContext to run their computation. While there is some trick to pass the ExecutionContext, it’s usually cumbersome and clutter the code.
In my opinion many of the problems come from a misunderstanding or misconception on how Futures work. (e.g. the strict nature of Futures and the way they interact with Execution Contexts to name a few).
In the previous post we’ve seen how to implement a gRPC service in Scala. While gRPC is a great way to implement remote services, many client/server interactions are still implemented using REST/HTTP nowadays.
So the question is: Is it possible to use gRPC to define and implement services and make them available over REST at the same time?
Well, it turns out it’s possible. The translation from protobuf to JSON format is quite straightforward. The only thing left is to define associated endpoints for each of the gRPC calls. Here again it can be easily done along with the gRPC definition using the HTTP annotations from Google.
This is exactly what GRPC-Gateway provides. While GRPC-Gateway is implemented in Go it can run along with any gRPC implementation as it only relies on the protobuf/gRPC definitions (.proto files). It then spawns up an HTTP proxy that translates and forwards REST (HTTP/JSON) calls to a gRPC server.
As many might think, gRPC doesn’t stand for “google Remote Procedure Call” but is a recursive acronym meaning “gRPC Remote Procedure Call”. I don’t know if you buy it but the truth is that is was originally developed by Google and then open-sourced.
If you’ve been in the IT for a while RPC doesn’t necessarily bring back happy memories. On the JVM it all started with RMI in the 90s. RMI was inspired by CORBA and suffered from a lack of interoperability as both the client and the server had to be implemented in Java. RMI was also particularly slow as Java serialisation is not a very efficient protocol.
Later in the 2000s came XML based RPC with XML-RPC and especially SOAP. Both of these formats address the interoperability as it no longer matters how the client/server are implemented. They only need to speak XML. However XML is still not an efficient protocol and communications remain slow.
SOAP provides an interesting definition language (WSDL – Web Service Definition Language) that can be used to generate service implementations.
gRPC addresses all these drawbacks. By default, it uses protobuf (Protocol buffers) for the service definitions and as its serialisation mechanism, which allows it to interoperate with many different languages while providing an efficient serialisation protocol. gRPC also takes advantage of HTTP/2 to add streaming capabilities.
Many people see Kafka as a messaging system but in reality it’s more than that. It’s a distributed streaming platform. While it can be used as a traditional messaging platform it also means that it’s more complex.
Akka actors fits nicely with DDD (Domain Driven Design) to design an application. E.g. It’s quite natural to model entities as individual actors who can be persisted, …
One of the key aspect in DDD is the notion of bounded context. A bounded context is simply a “self-content” component. It can interact with other components but it is coherent on its own. Each bounded context has its own domain model which belongs only to itself and should not leaked or be influenced by other bounded context.
Anti-corruption layers (aka translation layers or adapter layers) are used to enforce this principle. Basically their role is to translate the core domain objects into/from another domain that is used for communication or persistence.
Protocol Buffer (aka Protobuf) is an efficient and fast way to serialise data into a binary format. It is much more compact than Java serialisation or any text-based format (Json, XML, CSV, …).
Protobuf is schema based – it needs a description (in a .proto file) of the data structures to be serialised/deserialised.
On the JVM, protoc (the Protobuf compiler) reads the .proto description files and generates corresponding classes.
For Scala there is a very good sbt plugin “scalaPB” that follows the same process and generates case classes corresponding to the .proto files definitions.
The .proto files are an easy way to describe a protocol between 2 components (e.g. services). However there are some cases (e.g. writing to persistent storage) where the .proto files definition are just unnecessary and add superfluous complexity. (Who likes to read auto-generated code?).
In such cases it would be much easier to serialise an object directly into protobuf (using its class definition as a schema). Afterall this is what the protobuf java binding does: it serialises (auto-generated) java classes into protobuf binary format.
It’s been a while we haven’t covered any machine learning algorithm. Last time we discussed the Markov Decision Process (or MDP).
Today we’re going to build our knowledge on top of the MDP and see how we can generalise our MDP to solve more complex problems.
Reinforcement learning really hit the news back in 2013 when a computer learned how to play a bunch of old Atari games (like Breakout) just by observing the pixels on the screen. Let’s find out how this is possible! Continue reading “Reinforcement learning”
Logging has been around on the JVM for a while now. It all started with Log4J back in 2001. Log4J was the first logging framework and it is still around today (in its version 2). It provides a simple and efficient API (compare to System.out.println that was in use before).
Get a logger for a class
Use that logger to log messages
val logger = Logger.getLogger(classOf[MyClass])
logger.log(Level.DEBUG, "I am doing something right now")
logger.error("Oops, something went wrong", theException)
Today there are a few more frameworks on the JVM but they all provide similar APIs as Log4J:
JUL(2002): java.util.logging provides a standardisation of Log4J and of course provides a similar API
Commons-logging (2002): Apache project providing a façade over Log4J, JUL, … still the same API
SLF4J (2005): Another façade over Log4J (1&2), JUL, JCL, … no much changes in the API
Logback (2006): Brings structured logging with an API compatible (and similar) to SLF4J (and Log4J)
Log4J2 (2012): Rewrite of Log4J inspired by Log4J and Logback with improved performances. The API does not change much though.
As you can see the logging APIs available on the JVM haven’t changed much over the last 15 years. The most interesting additions are structured logging and the Mapped Dependent Context (MDC) as we shall see later.