From Start-Up to Scale-Up: Building a Data Team at a High-Growth Global Ed-Tech Company
summary
Company scales from ~8M to ~45M users and data grows from 4 TB to 185 TB warehouse while a first data group becomes a seven‑person team. Engineering uses ClickHouse + Kafka + DBT (plus Hex, Confluent, CockroachDB) to move event streams, build a homegrown reverse‑ETL feeding Pipedrive and to reduced report runtime to <1 minute. Automated product usage feeds sales (reported as a Game‑Changer) and an inferred‑dimension method detects LMS per district.
AI-written summary · about
chapters (11)
- 0:00Building a data team from scratch
- 2:17Company overview and growth
- 9:45Three major goals
- 11:10CockroachDB data ingestion
- 12:16CRM integration and reverse ETL
- 13:41Data transformation with DBT
- 15:07Automating usage reports
- 16:36Inferring LMS usage from events
- 18:01Team culture and principles
- 21:10Current challenges and future plans
- 22:56Q&A
related talks
29mBlueprints for Scaling: Practical Steps for Scaling Your Data Platform to Handle Exponential Growth
36mFrom Machine Learning Startup to Big Data Company
40mThe Tools Empowering 5000 Employees
33mAccidental DataOps: How we matured our organization with data, and how you can too!
34mTales of Data Architecture Evolution
07mFrom Scaling APIs to Scaling Data to Scaling Your Team
30mData Engineering building your BI infrastructure from scratch
42mWeb Development To Big Data. A Journey