Under the Hood: Engineering at William Hill
summary
464 bets per second, 5,000,000 daily price changes and 160TB/day flow through a 24x7 platform that uses Apache Kafka as the firehose to persist and replay streams, Docker for deployment and scaling, and ElasticSearch + Kibana dashboards for realtime inspection. A recommendations system runs in-memory as Erlang processes, computes similarities with the Jaccard coefficient, and demos a live slipstream; testing saturates 32 CPUs and demonstrates horizontal and vertical scaling with Docker for horizontal and vertical scaling.
AI-written summary · about
chapters (41)
- 0:00Introduction
- 0:30OAC
- 1:23Iona
- 2:29Disney
- 4:10Hollywood Tower
- 5:15Park Simulations
- 6:05William Hill
- 6:32Why William Hill
- 7:02Price changes
- 7:17Availability
- 7:34Fault Tolerance
- 8:14Data Handling
- 8:25Slipstream
- 9:22Trafalgar
- 12:01Trading Platform
- 12:43Kafka
- 13:17Architecture
- 13:58Settlement
- 14:47Hierarchy
- 15:47Process graph
- 17:03Scaling
- 18:03Dashboards
- 19:36Analytics
- 21:06LMK
- 22:20Application
- 23:25Provider
- 24:52Defaults
- 25:39GProc
- 26:41LinkBasic
- 26:58Sports Betting Recommendations
- 27:08Building
- 27:54Jacker Coefficient
- 30:07Weighted Recommendation
- 30:51Sports Mapping
- 32:29Good Pitches
- 32:53Platform
- 34:28This is Life
- 35:32Graph Memory
- 36:34Recommendation System
- 36:55Algorithms
- 37:08Graphs
related talks
39mScaling Betfair Exchange to support 160k rps using Apache Kafka
15mGrand National 2021: Managing Extreme Online Demand at William Hill
38mHow to handle 100k events per second
46mBeyond Betting: How PMU Reimagined Its Data Streaming Service Experience
28mSurviving the Grand National
28mMoving Towards Real-Time OLAP - With First Principle Thinking and Guided by Access Patterns
48mHyper Speed! When Big Data Blooms
46mEvents Analysis with Riak, Perl and Erlang at Booking.com