The 12 Factor App for Data
summary
Twelve concrete principles promise testable, deployable data platforms: use data structures as code (Flyway/liquibase/terraform), prefer append-only data structures for time travel, optimise retrieval (denormalise, S3 with schema plans), separate data from logic to avoid leaky abstractions, design for regression testability using immutable streams like Kafka/Redpanda/Flink/Pulsar, catalogue with OpenMetadata/Atlas, track lineage and code provenance, ban direct DB links and treat data as a product with SLAs and APIs.
AI-written summary · about
chapters (14)
- 0:0012 Factor App for Data intro
- 1:11Why 12 Factor for data
- 7:52Data structures as code
- 10:29Append-only data for history
- 13:24Optimizing data access
- 15:30Separate data from logic
- 22:07Strongly typed data
- 24:59Regression test data pipelines
- 29:13Data cataloging
- 31:00Data lineage tracking
- 32:49Code traceability in data
- 33:56Data domains and policies
- 37:56Data as a product
- 39:01Q&A
related talks
41mTwelve-Factor Apps
24mTwelve Factor App
27mCF & The 12-factor App 101: The Secret Decoder Ring for Operations
34m7 Missing Factors for Your Production-Quality 12-Factor Apps
49mThriving in the cloud: Going beyond the 12 factors
46mThriving in the cloud: Going beyond the 12 factors
42mThriving in the cloud: Going beyond the 12 factors
56mData Unleashed: A Developer's Perspective on Navigating the Architecture Maze