Storage Architecture for Yandex.Direct Advertising Objects

on YouTube ↗40m

" summary

Describes the architecture of Yandex.Direct’s storage for advertising objects, replacing MySQL by turning an in-memory hash map into a full database and using YTsaurus. Covers decisions on data normalization, streaming data processing, mandatory data versioning, and handling propagation of updates across object hierarchies.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 3:32Legacy architecture problems
3. 6:03GRUT central service
4. 8:38YDB tables and framework
5. 11:15Normalized vs denormalized
6. 13:49Event processing and scalability
7. 16:44Attribute groups and access control
8. 18:50Versioning for consistency
9. 22:10GRUT unified ad database
10. 23:15Benefits achieved
11. 25:02Q&A

" related talks

• 40mDatabases in the CloudVladimir Borodin · HighLoad++ Moscow 2018
• 49mSharding: from scratch to Yandex DiskAndrey Kolnoochenko · HighLoad++ 2023
• 42mPetabytes in YDB over HDD in Yandex.Metrica ProcessingAnton Barabanov · HighLoad++ 2023
• 48mYTsaurus — The Future of DWH, Already Here in Yandex MarketFilipp Kozmin · Saint HighLoad++ 2023
• 41mEfficiently Updating Database State from Stream Processing ServicesYuriy Pechatnov · Saint HighLoad++ 2023
• 44mInside S3Pavel Levdik · HighLoad++ 2023
• 54mYTsaurus: Operational Experience with a Storage Cluster of 180K DisksPavel Sushin · Saint HighLoad++ 2023
• 51mYandex Database: Distributed Queries in the CloudsSergey Puchin · Backend Conf 2019 (РИТ++)