Use AWS WAF to help avoid cost-prohibitive traffic in LLM apps
summary
Public-facing LLM apps see roughly 47% bot traffic and face model denial-of-service that drives inference costs up. A 1,200-word RAG prompt (~1,400 input tokens) plus a 150-token output costs about $0.006 per LLM request on Bedrock/Claude, but adding AWS WAF + CloudFront and AWS WAF Bot Control can block ~43% of malicious requests and—based on the example—translate to roughly $277 saved per $1 WAF spend, while CloudFront absorbs massive spikes (150M RPS).
AI-written summary · about
chapters (8)
- 0:00Threats to generative AI
- 2:19LLM fundamentals and RAG
- 6:55Costing LLM applications
- 8:14Securing with AWS WAF
- 14:28CloudFront protection
- 16:19Implementing WAF rules
- 16:48Observability and monitoring
- 17:41Summary and next steps
related talks
45mThe good, the bad and the verified - AWS WAF vs. bots
33mNetwork and application protection on AWS
53mHow AWS threat intelligence becomes managed firewall rules
25mWill LLMs really consume the Internet?
37mSecure by Design: Strategies for LLM Adoption in Cloud-Native Environm...
16mProtecting your application from bot attacks (PRT019)
17mWebLLM: A high-performance in-browser LLM Inference engine
27mSecuring Applications and Managing WAFs at a large scale