Groupe Rocher: how we made our CRM data platform greener?
summary
Builds a layered CRM data platform that reduces duplication and idle compute by enforcing serverless GCP stack (Pub/Sub, Cloud Functions, Cloud Composer, Dataflow, BigQuery, Cloud Storage) and a strict layered enterprise/business datasets model so enterprise datasets feed business datasets instead of copying data. Implements bucket lifecycle retention to delete raw files automatically and a short component shortlist to limit tooling sprawl. Begins measuring energy with GCP+Tableau and rolling out end-to-end retention and governance; platform is in production and being iterated.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:15Digital Sustainability Challenge
- 7:07Design the Data Platform
- 9:37Manage Your Data
- 9:53Processes
- 11:20Governance and Animation
- 12:20Understand Your Data
- 13:28Design of Hot Data Platform
- 14:00Three Business Issues
- 19:48Overview of Stack
related talks
37mBuilding a Data Platform from scratch
28mUsing Dataops To Build And Manage Data Products On Snowflake
41mFrictionless Compliant Data & Analytics Environments
33mBuilding an Elastic, Resilient Data Layer to Power New Retail Experiences
31mBe a Productive Dev with Real time Analytics as an API
41mModern Data Stack: Optimising and Scaling Data in a Tech Company
43mAnalytics capabilities architectural patterns: A journey for data-driven insights
01h 05mRoundtable: Cloud Databases as a Service