YTsaurus — The Future of DWH, Already Here in Yandex Market
summary
YTsaurus is presented as a replacement for traditional DWH stacks: the team needed a 30 PB storage with 10 PB/year growth and strict ingestion SLAs, found Greenplum + MapReduce inadequate, and adopted YTsaurus to meet those scale and latency requirements. The talk covers the practical migration and operational experience running YTsaurus as the Yandex.Market data warehouse.
AI-written summary · about
chapters (16)
- 0:00Data warehouse introduction
- 1:34Data warehouse history
- 3:01Warehouse parameters and issues
- 4:53Hadoop experience and YTsaurus transition
- 7:25YQL and YTsaurus benefits
- 10:41Reliability and calculation strategy
- 13:51Building detailed layer
- 17:22Normalization and denormalization
- 20:18Algorithms for detailed layers
- 23:06Scalability and testing
- 25:11YTsaurus cost and benefits
- 26:16Simplified warehouse architecture
- 27:18Experiment results
- 28:45Future of data warehouses
- 30:09Q&A
- 47:14Conclusion and demo
related talks
47mYTsaurus Dintables — yet another DBMS from Yandex
42mPetabytes in YDB over HDD in Yandex.Metrica Processing
40mStorage Architecture for Yandex.Direct Advertising Objects
39mTransactional Replication in YTsaurus
21mPreparing Yandex search data — the library and processes we built
01h 06mYahoo!, Big Data, and Microsoft BI: Bigger and Better Together
34mHow to drive a million kilometers in YTsaurus
40mDatabases in the Cloud