DataEngBytes 2023 - SYD-T1-04 - Yash Sharma
summary
Atlassian describes building a multi-region transactional storage platform that centralizes compliance and storage with a control plane, data plane, migration workflows and tenant routing, implemented on Amazon S3 for blobs and DynamoDB for entities, using Step Functions, Lambdas and Batch for migrations and BYOK encryption for revocable keys. The entity store already holds 2.4 billion entities and 3–4 TB live, projects to 20 PB next year, and the platform is reducing implementation time for 100+ services while handling tenant-level rate limiting and region migrations automatically.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 2:07Transactional data platform
- 5:16Data storage and compliance
- 8:28TDP architecture layers
- 12:21Platform ecosystem and Savvy
- 15:34Current scale and future
- 17:18TDP challenges and next steps
related talks
01h 03mReinvente su estrategia de datos con Lagos de Datos Transaccionales
46mKEEPING UP IN A DATA-INTENSIVE WORLD WITH SINGLESTORE
27mThe journey of Data Mesh and Compliance in a Multi-Regional Data Lake
51mImmutable data stores and CORS for safety, flexibility and profit
30mDataEngBytes 2023 - BRI-T1-08 - Ryan Boyd
11mBreaking Data Silos: Enterprise Data Management
30mBuild for Scale, Performance With Feature-packed Amazon DynamoDB
30mFrom monolith to petabyte scale: Built a cloud-native, AI-ready data layer