Experience report on using Akka in production: pitfalls and rewards
summary
A ScalaIO 2019 talk reports two years of running an Event Sourcing + CQRS application built with akka-persistence and akka-cluster-sharding, persisting events in Cassandra and publishing via Kafka. It covers practical ops: nightly load tests at 5,000 users, snapshots and migrations with Stamina, cluster formation using Consul/Zookeeper, short TTL (~30s) to reduce split‑brain, and lessons on deployment, scaling, monitoring, thread pools, serialization and event evolution; the cluster can auto-recover after failures.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 2:16Akka and you?
- 7:41Akka actors
- 9:41CORS, Event sourcing
- 14:07Application architecture
- 17:01Defining your protocol
- 18:21Actor management V2
- 19:10Actor façade
- 19:56Everything is asynchronous
- 20:29Pros and cons of futures
- 22:54akka-stream to the rescue
- 24:35Application initialization
- 26:01Event evolution
- 27:40Version evolution with stamina
- 28:57Deployment updates and high availability
- 29:11Docker swarm
- 30:10Cluster formation - V1
- 31:43Libraries for cluster formation
- 33:00Load scaling
- 35:07Fault tolerance / split brain
- 36:59Configuration example
- 38:21Monitoring
- 40:23Ease of adding metrics
- 40:41Handling capacity
- 43:10Points of attention
- 45:08Conclusion
related talks
41mAkka in Production: Our Story
48mThe three paths of Scaling with Akka
44mLong-lived microservices
51mImplementing Microservices with Scale and Akka
52mEvent Sourcing and CQRS
55mHow to fail at building a reactive information system? — MNT postmortem
48mEvent Sourcing - what could possibly go wrong?
59mImplementing Microservices with Scala and Akka