Cross Atlantic: Scaling Instagram Infrastructure from US to Europe
summary
Instagram partitions its stateful stores to avoid cross‑Atlantic quorum traffic: it uses partitioned Cassandra (replication reduced from eight total copies to 5 in US and 3 in EU) and moves users by social hash buckets fed from Akio, migrating buckets gradually to cut cache misses and latency. Hot global objects get a global handler; writes to Tao become local Tao writes that asynchronously replicate to the US master. This saves storage and latency but forces more Django capacity and separate DR sizing (per‑region headroom rises from ~20% to ~50%).
AI-written summary · about
chapters (6)
- 0:00Start
- 8:10Social Hashes Algorithm
- 9:18Why Social Hash Helped Us To Partition the Cassandra Dataset
- 12:31Cassandra Use Case
- 14:51Disaster Recovery Plan
- 20:41Region Failure
related talks
21mKeynote - Database Infrastructure at Instagram Scale
39mHow Geo-Distributed Apps Use Global Cloud Infra for Speed and Compliance
42mHacking The Pachyderm: Scaling Servers And People
03h 02mAPAC Live Stream
40mBut can you really run your app on 2 clouds at the same time?
51mData Residency: the fun of storing data worldwide
36mStories of Cassandra in Adobe Audience Manager
31mToo Far Too Furious: Multi-Region Cloud Applications