Use counters to count things
summary
Shows a 32‑bit counter overflow hitting exactly 2,147,483,647 rollover and fixed by switching to string formatting, demonstrates a costly log search over petabytes using EMR + SageMaker log scan after rejecting Athena at ~$5/TB (jobs run, die, required two weeks of babysitting but produce the needed counts), compares approaches: cheap 4–8 byte counters that are commutative and associative, naive hash sets, HyperLogLog (~2% error) and a newer coin‑toss sampling algorithm for distinct counts.
AI-written summary · about
chapters (11)
- 0:00Introduction and counting game
- 1:53Meet the speaker Fred
- 2:56New Year's Eve incident story
- 8:2032-bit integer rollover lesson
- 9:27Security incident counting challenge
- 13:40Counters vs log scans trade-offs
- 17:51Log delivery issues at scale
- 18:53Count distinct problem overview
- 20:18HyperLogLog explained
- 22:44CVM algorithm deep dive
- 24:50Q&A and closing
related talks
23mDesigning for High-cardinality Metrics
48mProbabilistic data structures
25mQuantiles at Scale: Choosing the Right Estimation Algorithms for Observability
37mWhen Probably is Good Enough
33mCPU Hardware Counters
43mPrometheus Puzzlers: Traps, Pitfalls, and Corner Cases
56mSpark-driven Audience counting
26mPetabyte Scale Datalake Table Management with Ray, Arrow, Parquet, and S3