Chapel Unblocked: Recent Communication Optimizations in Chapel

on YouTube ↗27m

" summary

Compiler and runtime work converts blocking network calls into faster paths and yields concrete wins: switching GASNet transfers >=4KB to its block-transfer engine saturates NIC bandwidth; non-blocking buffered task spawns remove round trips and give ~9x faster task creation at 1,024 nodes; compiler-emitted unordered atomic chain transactions plus buffering produce ~4–4.5× RandomAccess speedup (1.17→1.19) and a separate yield-tuning gives ~30% better atomics; Stream global variant reaches within ~10% of local up to ~25,000 cores.

AI-written summary · about

" chapters (24)

1. 0:00Intro
2. 0:30Performance Summary
3. 1:23Outline
4. 2:01Xscale
5. 2:42Timecraft
6. 3:05Eugenie
7. 3:59X
8. 4:20Stream
9. 5:47Stream Variations Performance
10. 6:25Task Creation
11. 7:55Communication Optimizations
12. 8:15Stream Performance
13. 8:58Random Access Improvements
14. 9:46MPI Calls
15. 10:54Memory Consistency Model
16. 12:49Results
17. 13:13Unordered Operations
18. 14:40Performance Improvement
19. 15:08Compiler Optimization
20. 16:27Performance Comparison
21. 17:22Summary
22. 18:03Whats next
23. 18:57Performance cliffs
24. 25:50Lunches

" related talks

• 27mFaster Than Fast: Networking and Communication Optimizations for Llama 3Adi Gangidi, Pavan Balaji · Networking @Scale 2024
• 17mBarre: Empowering Simplified and Versatile Programmable Congestion Control in...Zicheng Wang, Haohan Xu, Xiaoliang Wang, Zhuo Jiang, Xiaolong Zhong, Jianxi Ye, Haoran Wei, Yajuan Peng, Xiaoming Fu, Junkai Huang, Yang Bai, Huichen Dai · USENIX ATC '25
• 17mSupercharging Packet-level Network Simulation of Large Model Training via Memoization...Li Chen, Dan Li, Kaihui Gao, Fei Long, Yiwei Zhang, Fei Gui, Yitao Xing, Wenjia Wei, Bingyang Liu · NSDI '26
• 30mOn Sockets and System Calls: Minimizing Context Switches for the Socket APIHerbert Bos, Tomas Hruby, Andrew S. Tanenbaum, Teodor Crivat · TRIOS ’14
• 13mRakaia: Scalable In-Kernel Scheduling for TCP-Based RPCsRui Yang, Konstantinos Prasopoulos, Edouard Bugnion · 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26)
• 30mThe Gashlycrumb Tinies of AI Networking You Must Know (or Languish!)Lerna Ekmekcioglu · SREcon26 Americas
• 25mShenango: Achieving High CPU Efficiency for Latency-sensitive Datacenter WorkloadsJoshua Fried, Adam Belay, Amy Ousterhout, Hari Balakrishnan, Jonathan Behrens · NSDI 2019
• 41mPerformance Story: How to Speed Up HTTP/2 ClientSergey Kuksenko · Oracle Code One 2018