Drinking a river of IoT data with Akka.NET

on YouTube ↗59m

" summary

Demonstrates an Akka.NET architecture that turns each meter into a device-per-actor model, ingests events from IoT Hub / Event Hub via an App Service proxy, and routes messages over TCP remoting to a cluster with sharding and persistence. Shows normalization, timestamp bucketing and six gap‑fill strategies, persistent actors with persistence snapshots, and deployment flow that pauses ingestion to avoid data loss. Claims performance up to 50 million messages/sec per machine while showing practical persistence, recovery and monitoring patterns that work at scale.

AI-written summary · about

" chapters (48)

1. 0:00Introduction
2. 0:32Agenda
3. 1:03History
4. 1:45Ericsson
5. 3:16AkkaNET
6. 4:07What happened in 2015
7. 4:56The explosion of internet usage
8. 7:08Running parallel code
9. 7:52Amdahls law
10. 8:52Actor models
11. 9:44Actor
12. 10:53Actor code
13. 11:32Messages
14. 13:05immutable messages
15. 13:43actor system
16. 15:32actor hierarchy
17. 16:11supervision
18. 18:53risk to the edges
19. 19:17design patterns
20. 19:41character actor
21. 21:06IoT device
22. 21:22Meter readings vs consumption
23. 22:07Compute consumption
24. 22:39momentary consumption
25. 23:13overconsumption
26. 23:38iOS D stack
27. 25:14IoT Hub
28. 26:40Dont be a magpie
29. 27:42Local actor systems
30. 28:39Remote actor systems
31. 31:26Actor Proxy
32. 32:55Receive Actor
33. 35:00Device Actor
34. 37:06Data Loss
35. 38:02Timestamp Correction Bucketing
36. 39:09Filling Gaps
37. 39:29Flatlines
38. 41:26Normalized Messages
39. 42:18Data to Disk
40. 42:37Persistent Actor
41. 47:50Truncate Actor
42. 49:07System Restarts
43. 49:22Recreating Actors
44. 50:11Conclusion
45. 51:19Learning resources
46. 52:03Deployment
47. 52:31Conclusions
48. 53:05Questions

" related talks

• 52mDrinking a River of IoT Data with Akka.NETHannes Lowette · GOTO Copenhagen 2021
• 48mStreaming all the things with Akka StreamsJohan Andrén · Voxxed Days Zurich 2017
• 22mAdvanced Akka featuresJohan Janssen · Voxxed Days Vienna 2016
• 47mNetworks and Types — the Future of AkkaKonrad 'ktoso' Malawski · Scala Days New York 2018
• 15mAzure IoT from the edge to ADXJohan Ludvig Brattås · SQLBits 2023
• 31mFun with SensorsYogi Rampuria · Distributed SQL Summit Asia 2022
• 01h 32mFrom Spring + Java to Spring + Akka - A Journey of DiscoveryNilanjan Raychaudhuri, Josh Suereth · SpringOne2GX 2012
• 34mStream processing IoT time series data with RabbitMQ & InfluxDBSamantha Wang · RabbitMQ Summit 2021