Fifty Shades of Caching and How LLMs Paint It Blαck
summary
Industry-grade cache stack uses HAProxy, Varnish (memory), Apache Traffic Server (disk), PHP opcache/APCU, memcached behind mcrouter, and CDN edges to reduce origin load and terminate TLS. LLM/AI scrapers ignore robots.txt, distribute via residential/cloud proxies, and drive unpredictable spikes — Wikimedia reports 50% bandwidth increase since Jan 2024, dozens of distributed scrapers saturating POP links, and ~1.44M req/s served from caches while hosting ~1 PB of media. Mitigations include crawler bans, IP reputation, JS/cookie checks, and TLS fingerprinting.
AI-written summary · about
chapters (14)
- 0:00Caching and LLMs intro
- 1:46What caching is and why it matters
- 4:17Caching basics and browser cache
- 5:43Public caches and CDNs explained
- 9:39CDNs improve performance
- 14:00Application caching layers
- 15:47PHP in-memory caches
- 17:35Distributed caching with Memcached
- 20:25Cache freshness and invalidation
- 23:00LLM scrapers and traffic impact
- 24:47AI agents break caching
- 27:34Can we cache everything
- 28:59Fighting malicious scrapers
- 30:24Summary and future
related talks
56mCaching the uncacheable in Varnish
25mCaching strategies on https://dev.to
55mHow I brought LCP down to under 350 ms
47mCaching strategies on https://dev.to
28mCaching the uncacheable in Varnish
24mCaching Strategy for High Volume Personalized Content Requests
38mThe power of cache in a slow world
28mTaking Laravel to the edge with HTTP caching & Varnish