My Protobuf Module is Faster than Yours (Because I Cheated)
summary
Claims a custom Go protobuf decoder processes CrowdStrike traffic at about 35 million messages/sec by using aggressive tricks: pre-validated decoder objects, sync.Pool reuse of result structs and field buffers, and optional unsafe mode to alias []byte as string. Implementation trades memory-safety and complexity for performance, achieves near zero allocations at steady state, and outperforms standard decoders, but requires strict caller discipline and careful review because it can cause memory corruption or long-lived heap growth.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:27What is Protobuf
- 2:53Protobuf vs JSON size
- 3:38Dynamic Protobuf decoding
- 5:02Pre-validating Protobuf definitions
- 6:47Reusing allocations with sync.Pool
- 9:01Zeroing reused structs
- 11:08Unsafe string conversion
- 13:18Reusing slices for booleans
- 15:48Zero allocations steady state
- 17:13Benchmark results
- 17:34Why not in production
- 19:16Safety and trust
