YTsaurus: Operational Experience with a Storage Cluster of 180K Disks
summary
Describes operation of YTsaurus, Yandex’s big-data storage and processing platform used for ad-hoc analytics, ETL and batch jobs. Presents the challenges and solutions of running the largest YTsaurus cluster (over 20,000 hosts with varied configs and a total of 180,000+ disks), including operational practices and infrastructure approaches for such scale.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 6:03What is YTsaurus
- 12:30Working with data
- 16:29Basic monitoring
- 19:42Managing nodes
- 24:04Rolling updates
- 30:11Heterogeneous clusters
- 36:16Automating disk layout
- 39:46Conclusion & Q&A
related talks
34mHow to drive a million kilometers in YTsaurus
40mStorage Architecture for Yandex.Direct Advertising Objects
27mTales at Scale: Analytics at 1000 QPS and Beyond
01h 06mYahoo!, Big Data, and Microsoft BI: Bigger and Better Together
21mPreparing Yandex search data — the library and processes we built
30mAn architecture for big data: Supporting journalism through transparency in advertising
20mData in the Apache "big data" ecosystem
49mBuilding a 100‑Petabyte Analytics Warehouse