Data Mesh for the Cognitive City
summary
Argues that Data Mesh is the scalable architecture for a sensor‑dense cognitive city and shows a working stack: open‑source-first tools (Data Hub metadata catalog, GraphQL/REST), Huawei and Oracle clouds, Spark, Airflow, Terraform and Great Expectations. Demonstrates pushed metadata, end‑to‑end lineage from Kafka → Hadoop → Hive → Airflow outputs, and a live catalog UI. Runs on an active construction zone with 30,000 workers streaming wearables into a NeoSmash cognitive city OS and exposes reusable data products for downstream services.
AI-written summary · about
chapters (7)
- 0:00Introduction to speakers
- 1:42Why Data Mesh
- 7:46Data Mesh principles
- 13:04Cognitive City project
- 17:18Tooling for Data Mesh
- 20:18Demo of Data Hub
- 25:35Summary and resources
related talks
13mData Mesh is a Big Idea: Take Your Team From Vision To Reality
36mIntroducing Data Mesh A Paradigm Shift In Analytical Data Management Architecture
38mData Mesh, Now
48mData Mesh Paradigm Shift in Data Platform Architecture -
17mData Mesh: The Speedrun
20mData Mesh, Data Fabric & Co - the End of Monolithic Architecture Concepts?
51mData Mesh – Successful with People, Process, and Technology
39mData Mesh: Why, What and How