From Chaos to Order: How to Konnect, secure and observe [...]
summary
Kong’s AI Gateway proxies LLM calls, keeps provider keys centrally, enforces OAuth2 authorization and per-client token rate limits (demo triggers 429 when limits hit) to stop budget spikes; it applies semantic firewalling to block topics like politics, performs privacy sanitization, and uses semantic caching (vector DB) to cut downstream token calls. REST APIs convert to MCP tools, metrics export to Prometheus/Grafana/Elastic, integrates with Volcano SDK and OpenTelemetry, and a fintech report claims 60% cost reduction in production.
AI-written summary · about
chapters (26)
- 0:00AI gateway challenges
- 3:40REST vs AI APIs
- 4:44AI gateway as universal API
- 6:27Semantic firewalling
- 7:35Sanitizing private data
- 8:20Cost control with rate limiting
- 9:03Semantic caching
- 10:12Observability and analytics
- 10:35Demo: new AI model
- 14:37Guardrails for policy
- 15:22Demo: guardrails testing
- 17:10Token limits for cost
- 19:20Model Context Protocol
- 21:04REST to MCP servers
- 21:45Demo: API to MCP
- 24:14Testing MCP server
- 27:12LLM usage dashboard
- 28:41Authorization and access
- 29:24Observability integration
- 30:31AI agentic workflows
- 32:45Volcano SDK
- 34:51Demo: simple AI agent
- 35:59Demo: multi-model agent
- 38:37Metrics with Prometheus
- 39:18Real-world benefits
- 40:03Conclusion
related talks
45mRun, Secure, and Govern Your AI Traffic with an AI Gateway for Multi-LLM
17mOperationalizing API Management at Scale on India’s Enterprise Frontlines
08mObserving Agentic/MCP Traffic & Keeping Costs Low
25mOAuth Gets You In, Zero Trust Keeps You Safe
45mAPI Security: Acting Now to Enable Security Controls
36m10 Things You Hate About Your API Gateway
18mGateways are All You Need
39mAPI Security: Acting Now to Enable Security Controls