Leveraging Scala And Akka To Build A Distributed Time-series Database

on YouTube ↗44m

" summary

NSDB is a Scala time-series DB built on Akka: it uses natural time sharding with configurable shard intervals and Lucene-backed shards, routes writes through an actor hierarchy (accumulator → performer) and exposes gRPC/HTTP/WebSocket APIs plus Flink and Kafka connectors. A multimaster peer-to-peer cluster replicates data with configurable consistent vs eventual replicas (writes sync to consistent replicas, async to eventual), uses a custom commit log, an unbounded control mailbox for publish latency, and a Saga pattern for distributed write coordination; single-node throughput reaches millions of records and the project is Apache‑2 on GitHub.

AI-written summary · about

" chapters (31)

1. 0:00Introduction and overview
2. 1:08Talk structure and agenda
3. 2:11Why NSDB was developed
4. 3:13Time-series data model explained
5. 5:00Consistency model in NSDB
6. 6:50NSDB in CQRS architecture
7. 7:34Natural time sharding
8. 9:03Write operations in NSDB
9. 10:08APIs and connectors
10. 11:34Publish-subscribe mechanism
11. 12:16Single node design overview
12. 13:20Akka actors fundamentals
13. 14:23NSDB tech stack
14. 15:05Storage layer with Lucene
15. 16:10SQL-like query support
16. 17:33Actor hierarchy in NSDB
17. 19:22Write flow in single node
18. 20:02Read flow in single node
19. 21:31Data streaming flow
20. 23:20Akka Cluster overview
21. 25:03Akka Distributed Data
22. 26:53Akka Distributed PubSub
23. 27:36Distributed architecture
24. 30:05Replication strategies
25. 32:41Write to replicas process
26. 33:45Read from replicas process
27. 35:32Distributed transactions
28. 36:13Saga pattern for rollback
29. 37:59Roadmap and future features
30. 38:43Open source and licensing
31. 39:30Q&A session

" related talks

• 40mBuilding a High-Performance Database with Scala, Akka, and SparkEvan Chan · Scale By the Bay 2017
• 36mScale By The Bay 2018 Unconference: Rob NorrisRob Norris · Scale By The Bay 2018
• 33mScaling Financial Automation on TypeBusKaoru Kohashigawa, Tabitha Blagdon · Scale By The Bay 2019
• 43mBuild a Modern, End-to-End, Big Data Scala Reference ApplicationEdgar Orendain · Scale By the Bay 2017
• 44mWriting a Database in Scala.jsRyan Peters · Scala Days 2023 Seattle
• 44mBuilding event-driven microservices with Scala and Functional Domain ModelsChris Richardson · Great Indian Developer Summit 2015
• 22mAdvanced Akka featuresJohan Janssen · Voxxed Days Vienna 2016
• 42mBuilding a Company on ScalaDag Liodden · Scala Days Copenhagen 2017