Compose :: Melbourne will feature many excellent speakers. One of this year's lineup is Luke Stephenson. If you want to see the whole lineup look here!
This talk introduces Reactive Streams, specifically the Monix implementation and how at REA Group we have used it to solve real problems.
At REA we have many 'feeders' which extract from one data source, perform some processing and transformations before loading into a target system.
Historically these feeders have been built using Java / Scala concurrency primitives (e.g. Future / ExecutionContext). The typical feeder has a very sequential execution model like:
The very sequential nature means the feeds are not as performant as they could be. The source and target systems were never accessed in parallel. Writing to the target required waiting for the source to read another batch. And the feeder would not read another batch from the source until the target completed.
Also being based on lower level concurrency primitives, the data pipeline is not as easy to visualise from reading the codebase.
Reactive streams allow us to model asynchronous processing pipelines as a data type which can then be executed. The APIs are declarative, as opposed to the legacy solution working with Scala Future and ExecutionContext which resulted in immediate evaluation.
The talk will cover:
Attendees should leave with a basic understanding of Reactive Streams, and specifically how to get started with the Monix implementation of reactive streams. I'll share a working basic project (not REA production code) which provides a basic 'feeder' structure for attendees to reference and extend upon if they wish.