Changing a Tire on a Moving Car: Migrating to Datadog’s New Event Store without Downtime

on YouTube ↗37m

" summary

Datadog built Husky, a third-generation event analytics database, to support petabyte-scale event data and features the legacy system couldn’t handle. The talk covers implementing late data ingestion and queries over unfaceted fields by redesigning the event store, and explains how Datadog migrated thousands of customers across multiple products to Husky with zero downtime.

AI-written summary · about

" chapters (8)

1. 0:00Start
2. 3:47Feature requests
3. 5:42Filtering/aggregating on all attributes
4. 8:05How many columns can we support?
5. 8:39Sparse Column Encoding
6. 9:43Late Data Ingestion: Husky
7. 16:43Migration Progress
8. 17:22Takeaways

" related talks

• 36mHot Swap Your Datastore: A Practical Approach and Lessons LearnedRaj Shekhar, Mehmet Can Kurt · SREcon20 Americas
• 25mManaging expectationsSeshendra Nalla · LeadDev New York 2024
• 01h 41mDatadog DASH 2019 KeynoteAparna Sinha, Alexis Lê-Quôc, Michael Gerstenhaber, Justin Wright, Calvin French-Owen, Miranda Kapin, Brent Montague, Celene Chang, Vishesh Sharma, Renaud Boutet, Kartik Garg, Amit Agarwal, Priyanshi Gupta, Andrew Southard, Gabriel-James Safar, Hugo Kaczmarek · Dash 2019
• 31mModernizing a megascale databaseToua Her, Vishal Prabhu, Sudarsanan Ranganathan · AWS re:Invent 2022
• 51mImmutable data stores and CORS for safety, flexibility and profitSidney Shek · Scala Days Berlin 2016
• 35mBuilding a Real Time Metrics DatabaseJoel Barciauskas · Dash 2019
• 47mData Warehousing Migrations: Lessons from Home DepotTino Tereshko · Google Cloud Next '18
• 22mMigrating from Monolithic to Cloud Native Distributed SQL Database with Zero DowntimeSUHENDRA AHMAD · Distributed SQL Summit Asia 2022