How to drive a million kilometers in YTsaurus
summary
YTsaurus is presented as a distributed data-processing platform; the talk explains its map-reduce operation and how it runs a large-scale autonomous-vehicle driving simulation. It covers performance pitfalls that slow simulation, techniques for faster input data reads, and methods to deliver application code more efficiently.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 2:13Real-world metrics challenges
- 4:03Need for fast simulator
- 9:47Simulation pipeline
- 16:53Developer workflow
- 19:24Simulation setup evolution
- 25:59Optimizing deployment speed
- 27:03Q&A
related talks
54mYTsaurus: Operational Experience with a Storage Cluster of 180K Disks
39mTransactional Replication in YTsaurus
48mYTsaurus — The Future of DWH, Already Here in Yandex Market
40mYTsaurus SPYT: Helping the Apache Spark Scheduler Be Even More Efficient
11mTowards a Digital Society: Investigating Integration of Distributed DBs and Distributed Processing
22m"How We Built a Rust-y Real-Time Public Transport Map"
47mpython at scale – creating a High Definition Map of the world's road network
37mThe global data plane for autonomous driving from Day 0 to Day 2 — Stephan Reimann and Simon Putz