Business Intelligence in Microservice architecture #bbuzz
summary
Bol.com moves BI from a monolithic warehouse to a hybrid microservice model using Hadoop/HBase (26 nodes, ~1.5 PB), Flink for streaming, and Sqoop/Pig for batch; a custom edge-based tables pattern with time-prefixed keys and bulk interfaces (event-sourcing style) lets services replay deltas, reducing some slow offer jobs from ~24 hours to near-real-time hours. Built tooling (Eddie to ingest, Platora and a Flink wrapper, Docker+Mesos deployment) that works at scale but needs a dedicated ops team and careful handling because Flink and stream ops are still brittle.
AI-written summary · about
chapters (10)
- 0:00Introduction and background
- 4:42What is business intelligence
- 7:57Traditional BI in monolithic systems
- 11:11BI challenges in microservices
- 12:59Introducing bulk interfaces
- 15:32Delta processing in Hadoop
- 18:02Self-service BI limitations
- 21:59Transition to streaming architecture
- 25:59Deploying Flink on Mesos
- 27:27Q&A session
related talks
41mBig data as we know it @bol.com
22mData integration in the world of microservices
47mApplying “web scale” patterns (...)
32mNubank's experience scaling microservices operations
18mMicroservices Out In the Wild
42mApache Flink deep-dive #bbuzz
31mSharing is caring: Building a modern microservice architecture helps everyone get ahead
57mOur DataMesh Journey: data architectures for microservices