Immutable data stores and CORS for safety, flexibility and profit
summary
Atlassian is rewriting its identity platform using event sourcing plus CQRS, capturing immutable JSON events in DynamoDB and replaying them into targeted stores; production dogfooding shows parts running ~18 months and gives audit trails for free and safer zero‑downtime schema changes. Implementation uses scalaz‑streams/FS2, DynamoDB streams to Kinesis and AWS Lambdas, plus a query synchronizer that writes to Postgres and can batch or force reads using opaque sequence tokens for consistency. Design choices: tiny set‑payload events, per-user event streams for parallelism, check‑and‑set or CRDT resolution to balance consistency and latency.
AI-written summary · about
chapters (19)
- 0:00Start
- 4:27Our Identity System requirements
- 11:49Modeling Events
- 12:40Event storage API
- 13:55Saving events
- 14:47Operation: Constraint as a type
- 16:03Querying event streams
- 17:55Query synchroniser is a scalaz-stream Channel...
- 20:41Insert / Update Delta
- 24:01Single stream
- 24:56Rules for splitting streams
- 27:15Query views get populated eventually
- 28:53Why do we need transactions?
- 29:47CAP or PACELC?
- 30:50Check-and-Set writes
- 32:51Tokens to emulate transactions and consistency
- 33:51Using tokens to enforce state
- 35:06Resolve conflicts on query
- 36:38Key takeaways
related talks
01h 25mBuilding microservices with event sourcing and CQRS
48mEvent Sourcing - what could possibly go wrong?
47m8 years of EventSourcing/CQRS @ bettermarks: The good, the bad and the ugly
40mBeyond Relational: Data storage for modern applications
52mData storage for modern applications - Mike Lehan
46mState or events? Which shall I keep?
36mImmutable Data Science with Datomic, Spark and Kafka
52mEvent Sourcing for the Cloud Developer