How Bloomberg Creates a Resilient Data Analytics Platform Using Karmada
Talkat KubeCon + CloudNativeCon China 2025 · on Wednesday, 2025-06-11· at 11:45· Level 19 | Crystal Court II·
summary
Bloomberg runs Apache Flink, Spark, and Trino across multi-cluster Kubernetes using Karmada to provide a unified submission endpoint, cluster-aware scheduling, and automated failover. Engineers add a CRD interpreter to compute Flink replica counts and health, implement state preservation via job ID with a webhook that injects the latest checkpoint on failover, and enable federated quota enforcement (example: 20 CPU, 50 GB) so failover succeeds; features land in Karmada v1.12 and v1.14 and are proven in production.
AI-written summary · about
chapters (9)
- 0:00Bloomberg's data analytics platform
- 2:27Multi-cluster data platform challenges
- 4:56Karmada for cross-cluster workloads
- 5:18Flink integration with Karmada
- 17:17Trino on Karmada for querying
- 21:25Spark workloads on Karmada
- 28:48Karmada benefits for Bloomberg
- 30:35Karmada community and future
- 32:26Q&A
related talks
25mFlink on Karmada: Building Resilient Data Pipelines on Multi-Cluster K8s
04mProject Lightning Talk: Karmada Updates and Use Cases
24mTaming Observability at Scale in a Multi-Cluster Kubernetes Platform at Bloom...
05mMulti-Cluster Orchestration System: Karmada Updates A...
34mMulti-cluster Orchestration System: Karmada Updates and Use Cases
30mSimplifying Multi-cluster Kubernetes Management with Karmada
07mProject Lightning Talk: Multi-cluster Orchestration System: Karmada Updates and ...
30mSailing Multi Cloud Traffic Management With Karmada