~/talk

Blueprints for Scaling: Practical Steps for Scaling Your Data Platform to Handle Exponential Growth

Talkat DataEngBytes Perth 2024 · on Friday, 2024-09-27· [data-engineering]
links:none on file
slides:none on file

Canva’s data platform grows to ~25 PB with ~400–450 contributors and ~7,000 PRs/year; platform teams build an internal provisioning tool called Tundra for Snowflake, adopt dbt, FiveTran and Bigeye, and implement CI checks (Buildkite-style) that prevent breaking changes, saving hundreds of thousands of dollars in remediation. The team decentralizes into domain-aligned teams vs a monolith, provides composable primitives and paved-road modules, and slashes PR environment spin-up from 45 minutes to under 5 minutes, driving faster time-to-insight.

AI-written summary · about

  1. 0:00Introduction
  2. 1:08Scaling pains in data platforms
  3. 2:08Canva's platform growth
  4. 4:16Aligning platforms to business value
  5. 8:10Choosing platform topology
  6. 12:21When to use data mesh
  7. 15:28Composable components for platforms
  8. 18:59Driving adoption with incentives
  9. 23:53Measuring platform performance
  10. 27:49Recap and scaling analogy
~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.