FlexTOE: Flexible TCP Offload with Fine-Grained Parallelism

on YouTube ↗14m

" summary

FlexTOE is a software TCP offload engine that runs on SmartNICs (Agilio‑CX40, Mellanox BlueField and x86 prototypes) and eliminates host TCP data-path by streaming payloads without NIC buffering. It uses a fine-grained parallel pipeline with per-module sharding and reordering, saves up to 81% host CPU versus Chelsio Terminator, scales Memcached throughput 38% better versus TAS, cuts 99.99th‑percentile RPC RTT 3.2× versus Chelsio and 50% versus TAS, and supports POSIX sockets plus C/eBPF XDP extensions.

AI-written summary · about

" chapters (11)

1. 0:00Intro
2. 0:26High CPU Overhead of TCP
3. 1:20Need for Flexible TCP Offload • FlexibilityDatacenter networks evolve rapidly
4. 3:11TCP Offload to SmartNICs - Challenges
5. 5:52FlexTOE Offload Architecture
6. 6:42Parallelizing the TCP Data-path for Offload
7. 7:48Parallel TCP Processing Example: Transmit (TX)
8. 10:06Evaluation Setup
9. 10:39Benefits of Offload: Throughput Scalability
10. 12:07Is Fine-grained Parallelism Necessary?
11. 12:48Data-path Parallelism: Does it Generalize across Platforms?

" related talks

• 21mRemote TCP Connection Offload and ApplicationsMichio Honda, Shuo Li, Steven Chien, Tianyi Gao · NSDI '26
• 17mRearchitecting the TCP Stack for I/O-Offloaded Content DeliveryYoungjin Kwon, Minlan Yu, KyoungSoo Park, Taehyun Kim, Deondre Martin Ng, Junzhi Gong · NSDI 2023
• 21mAccelTCP: Accelerating Network Applications with Stateful TCP OffloadingKyoungSoo Park, YoungGyoun Moon, SeungEon Lee, Muhammad Asim Jamshed · NSDI 2020
• 14mHigh-throughput and Flexible Host Networking for Accelerated ComputingChristos Kozyrakis, Athinagoras Skiadopoulos, Mark Zhao, Shrijeet Mukherjee, Saksham Agarwal, Qizhe Cai, Rachit Agarwal, Zhiqiang Xie, Jacob Adelmann, David Ahern, Carlo Contavalli, Michael Goldflam, Vitaly Mayatskikh, Raghu Raja, Daniel Walton · 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24)
• 22mAdapting TCP for Reconfigurable Datacenter NetworksSrinivasan Seshan, Daehyeok Kim, Alex C. Snoeren, Weiyang Wang, Matthew K. Mukerjee, Christopher Canel · NSDI 2020
• 14mMultitenancy for Fast and Programmable Networks in the CloudXin Jin, Aurojit Panda, Hang Zhu, Tao Wang, Dan R. K. Ports, Anirudh Sivaraman, Fabian Ruffy · HotCloud '20
• 24mOmniXtend cache coherence protocol for datacenter CPUsZvonimir Z Bandić · ORConf 2019
• 19mConntrack offload: why and howMarcelo Ricardo Leitner · DevConf.CZ 2021