Context Aware Traffic Routing for Stateful Services using Envoy

on YouTube ↗17m

" summary

Retrofits per-request routing by peeking into protobuf payloads with an Envoy custom filter that does partial protobuf parsing to extract envelope fields (app_id, server_id, player_id) and route before full deserialization. Prototype reduces memory by >50% for small messages and ~30% for 2MB messages, shifts client→server mapping into Envoy caches coordinated by a state coordinator service, and preserves 100% correctness while cutting backend storage lookups and end-to-end latency.

AI-written summary · about

" chapters (15)

1. 0:00Thank you to our session recording sponsor!
2. 0:50Context aware routing We can route messages based on the context of the message Context is provided by: . URL routing (usually used by websites) • Request Header leg used by load balancers • Side channel (usually used by apps) . As part of the message content
3. 1:39Use cases • Retrofitting protocol based on content without application being aware • Use existing content to perform smarter routing
4. 2:44Custom filter overview
5. 3:08Load test setup Test Message size average OS KB
6. 4:48Protobuf Wireformat
7. 7:05Things of Note • Envelope can be retrofitted ., envelope does not have to be . The unique string can be changed to indicate backwards Oncompatible changes . More than one kind of address can be used for routing purposes
8. 7:35Things of Note • Envelope can be retrofitted ., envelope does not have to be . The unique string can be changed to indicate backwards Incompatible changes . More than one kind of address can be used for routing purposes
9. 8:02Load test setup (V2) Test
10. 9:06Test 1 Resource Utilization (V2)
11. 10:04Stateful message routing
12. 10:51Stateful services message routing
13. 12:14Proposed solution
14. 15:31Client server mapping coordinator
15. 16:10Conclusions • Shifting the client to server mapping can help with improving • Partial message deserialization can help when dealing with large message

" related talks

• 21mDeep Dive: EnvoyLizan Zhou · KubeCon + CloudNativeCon Europe 2019
• 28mAuthorization with EnvoyJelle Vanhorenbeke · EnvoyCon 2020 Virtual
• 42mAccelerating Envoy with the Linux KernelThomas Graf · KubeCon + CloudNativeCon Europe 2018
• 21mMaking Envoy Resilient to Sudden Increases in LoadBoteng Yao · EnvoyCon NA 2024
• 36mSecuring Every BitDavid Bell, Ronak Jain · EnvoyCon 2025 Europe
• 11mLightning Talk: Building a Scalable Multi-Protocol API Gateway with EnvoyMatt Poegel · EnvoyCon NA 2024
• 26mLeveraging Envoy to Implement Micro-Segmentation-Based Security PoliciesHermann Lueckhoff · EnvoyCon North America 2022
• 23mUsing Envoy as an Egress Proxy for TLS Enabled TrafficAmit Jain, Kiran Kumar · EnvoyCon North America 2021