Evolution of serialization formats and usage in Go
summary
Run-down of actual results and tradeoffs: compares net/rpc to protobufs + gRPC for polyglot microservices, inspects Go’s encoding packages and shows where easyjson code generation beats reflection, and explains why gob streams are self describing (first-send type schema then IDs). Covers Avro for Kafka/data pipelines, FlatBuffers for zero-copy game reads, Cap’n Proto for CPU-aligned parsing, and SBE for ultra-low-latency finance, with benchmarks, tooling and schema-evolution details and concrete deployment use cases.
AI-written summary · about
chapters (25)
- 0:00Intro
- 1:23Abstract
- 3:24History
- 4:14XDR
- 5:03Length Value Encoding
- 5:43Go Source Encoding
- 7:08Selection considerations
- 8:07What tooling is available
- 8:40How do you reduce overhead
- 10:25Data types
- 10:52Go encoding
- 11:38JSON
- 14:13JSONRPC
- 15:15GoBs
- 17:10Protobufs
- 18:29GRPC
- 19:49Protobuf
- 20:10Avro
- 20:45IETF
- 21:34FlatBuffers
- 22:59Captain Proto
- 24:25Simple binary encoding
- 25:09Do you need it
- 25:19serialization benchmarks
- 25:34question to the audience
related talks
42mgRPC and Go: Developing Efficient and Type-Safe Services
07mGRPC & Go
35mBeyond JSON: Improving inter-app communication
25mRethinking Data Flow in gRPC-Go: Maximizing Memor...
26mDemystifying Protocol Buffers and gRPC
56mEverything is serialization
26mComparing Serialization Formats and Libraries
24m"Building Scalable Microservices with Go"