Fast IP Address Matching in Elixir (...) & Persistent Term
summary
Solves matching 1 million IP prefixes by parsing CIDR into bit-keyed structs and storing them in a radix tree so lookups stay in microseconds versus list scans that become 5,000× slower at 10k and ~33 ms at 100k. Avoids a GenServer bottleneck and ETS because ETS forces a ~28 MB memory copy per caller, causing memory spikes (example: ~3 GB with 20 processes). Uses persistent_term to publish the radix tree once, keeping memory flat for many readers; updates trigger global GC and are expensive.
AI-written summary · about
chapters (4)
- 0:00Start
- 1:59Kat and Black Cat Concert Tickets
- 2:59Problem 1: Fast Matching on IP Prefixes
- 8:59Textbook example from getting started
related talks
28mGeolocation EXPLAINed
06mThumbnail Generation in Elixir/Erlang with libVIPS
01h 00mHyper Speed: When Big Data Blooms
43mCaching half a billion user transactions
33mProcessing 2.7 million images with Elixir (vs Ruby)
26mIP Clusters - Scaling Zopim's Frontend Node Servers
31mBetter Bandwidth Management with eBPF
47mMaking it Fast - How Zotonic Maximizes Erlang's Power