ClickHouse Without Pain: Our Experience Refactoring a Legacy Cluster and the Impact of Shard Groups

on YouTube ↗47m

" summary

Practical lessons from refactoring a legacy ClickHouse cluster: common mistakes in building and operating ClickHouse, the negative consequences observed, and concrete ways to avoid them. Also covers rolling out shard groups (shard_group feature) and the operational effects seen after that change.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:09Data warehouse architecture
3. 2:57ClickHouse implementation errors
4. 6:32Reasons for new ClickHouse
5. 7:34Post-migration solutions
6. 8:37Refactoring problems
7. 9:17Shard groups in ClickHouse
8. 13:11Distributed table behavior
9. 14:15Gradual shard-group transition
10. 15:44Refactoring results
11. 16:48Monitoring and metrics
12. 18:58Data volumes and load
13. 20:01Q&A
14. 41:30Fault tolerance and replication
15. 43:39Bad query monitoring
16. 45:05Cloud and infrastructure cost

" related talks

• 59mEffective Use of ClickHouseAleksey Milovidov · HighLoad++ 2017
• 51mWhat You Need to Know About ClickHouse ArchitectureAleksey Zatelepin · РИТ++ 2017
• 53mClickHouse Is Slowing DownKirill Shvakov · HighLoad++ Moscow 2018
• 53mClickHouse capabilities for advanced developersAleksey Milovidov · Backend Conf 2018
• 42mOptimizing ClickHouse query performanceMaksim Kita · Saint HighLoad++ 2022
• 53mClickHouse: blitz meetup (questions and answers)Aleksey Milovidov · HighLoad++ Moscow 2018
• 51mHow to Survive a One-Year Sprint and Migrate to ClickHouseDenis Trayzhon · HighLoad++ 2023
• 49mFlagrant Bugs and How to Avoid Them: The ClickHouse ExampleAleksey Milovidov · HighLoad++ 2019