Run your Business with Real-time Insights from Data
summary
GO‑JEK moves from day‑old reports to minute‑fresh analytics by piping events through Pub/Sub, processing and windowing with Dataflow, and writing results into BigQuery. Pipelines handle >10 billion driver pings (every 10s), autoscale workers, produce a live supply‑vs‑demand heatmap refreshed every minute, and generate cancellation alerts via Stackdriver and merchant notifications. Platform growth is 30% monthly with ~700 datasets and real‑time reports delivered within hours rather than days.
AI-written summary · about
chapters (11)
- 0:00Introduction to real-time insights
- 1:06Stream processing architecture basics
- 7:54Simple streaming to BigQuery
- 12:31Fixed and sliding windows explained
- 16:27Session windowing for user behavior
- 18:57Machine learning in streaming pipelines
- 25:46Stateful streaming for dynamic decisions
- 27:55GO-JEK real-world use cases
- 33:46GO-JEK supply-demand analytics
- 38:30Cancellation alerts in real time
- 42:24Key takeaways and Q&A
related talks
26mDemocratizing Data at Go-JEK
21mPutting big data self-service analytics in the hands of business people
24mData analytics with distributed tracing data
48mUsing Open Source Tech to Swap Out Components of Your Data Pipeline
55mCloud Analytics: from 0 to 100,000 events per second in 50'
24mGive meaning to 100 billion analytics events a day, analytics at Teads
53mIntroduction to data streaming
33mAccidental DataOps: How we matured our organization with data, and how you can too!