Storage, Then and Now

on YouTube ↗35m

" summary

History of Twitter storage turns into two concrete recipes: a custom internal system called Manhattan built ~6 years ago to be a multi-tenant self-service database that becomes the canonical store for tweets DMs users and ads, optimized for worst-case predictable performance under faults; and FaunaDB, a JVM/Scala packaged transactional DB aimed at cloud SaaS customers with strong consistency, global replication and fine-grained QoS. Both reject Cassandra’s trade-offs; both run on physical clusters, not containers, and both prioritize predictable latency over peak throughput.

AI-written summary · about

" chapters (8)

1. 0:00Introduction and welcome
2. 1:12Early storage at Twitter
3. 5:08Why not Cassandra
4. 9:46Message buses and databases
5. 14:07Managed database design
6. 19:28Operational and performance challenges
7. 25:28Open sourcing Manhattan
8. 30:46Future challenges and closing

" related talks

• 30mThe Distributed Database Behind TwitterMehrdad Nurolahzade · Distributed SQL Summit 2020
• 34mBig and Small Data at @TwitterJeremy Cole · O'Reilly MySQL Conference & Expo 2011
• 28mAn Architecture to Tweet Them AllJesús Sánchez · PyData Madrid 2016
• 34mData Storage: NoSQL Toasters and a Cloud of Kitchen SinksCasey Rosenthal · RailsConf 2013
• 40mProviding Flexible Database Consistency Levels with Manhattan at TwitterBoaz Avital · GOTO Chicago 2016
• 27mCloning Twitter: Rails + Cassandra = Scalable SharingCharles Max Wood · Rocky Mountain Ruby 2011
• 52mData storage for modern applications - Mike LehanBeyond Relational · PHP UK Conference 2024
• 05mBret Piatt interviewed at Velocity 2010Bret Piatt · Velocity 2010