Haxl A Big Hammer for Concurrency

on YouTube ↗48m

" summary

Haxl flips the default from sequential to concurrent by turning IO into data and using an applicative-do compiler extension so independent effects run automatically, with a user-supplied fetch implementation that batches or forks requests. It adds a per-request cache that makes recorded runs reproducible for unit tests and debugging. Deployed in production, it runs on thousands of machines serving over a million requests per second across hundreds of thousands of lines of Haskell and reduces latency while handling hundreds of changes per day.

AI-written summary · about

" chapters (57)

1. 0:00Intro
2. 0:33Slow IO
3. 1:07Real world changes
4. 1:57Problems are related
5. 2:16What are you talking about
6. 2:31Every language has some way
7. 3:15Theres a bunch of problems
8. 3:47Refactoring
9. 4:32Side effects
10. 4:56No side effects
11. 8:36Fake World
12. 8:47Mock API
13. 10:13Debugging
14. 10:31Logging
15. 12:08Big Hammers
16. 12:52Distributed Source Control
17. 15:14What is Haxl
18. 15:33Example
19. 16:08Get Latest Version
20. 16:41Get Hosts
21. 16:51Get Version
22. 17:06Update Version
23. 17:24Script
24. 19:22Dependencies
25. 20:05Parallelization
26. 20:23How does it work
27. 21:07Haxl vs Garbage Collection
28. 21:54Concurrent
29. 22:26How does concurrency work
30. 25:19Sequential
31. 25:31Applicative
32. 26:44Explicit Concurrent
33. 27:12Applicative Operator
34. 29:11Dependency Graph
35. 30:15Limitations
36. 31:02Refactoring the code
37. 33:20Haxl update
38. 33:51New contract
39. 34:41Batching tradeoff
40. 36:08Recap
41. 37:53Data is malleable
42. 38:03Internal cache
43. 38:41Modularity
44. 38:58Caching
45. 40:03Cache
46. 40:35Unit Tests
47. 40:59Cache Request
48. 41:13Synthetic Test Data
49. 42:18Why we made Haxl
50. 42:34Sigma
51. 43:17Haskell
52. 43:45Performance
53. 44:07Stats
54. 44:35Github
55. 44:47Hacks
56. 45:24Summary
57. 45:48RunAxle

" related talks

• 01h 02mHPX - A C++ Library for Parallelism and ConcurrencyHartmut Kaiser · CppCon 2022
• 01h 06mAsynchronous Computation in C++Hartmut Kaiser · CppCon 2014
• 46mUniversal Highload Patterns: A Game Server Example in HaskellMaksym Bezuglyi · Highload fwdays'18
• 15mTowards Safe HPC: Productivity and Performance via Rust Interfaces for a Distribu...Jeffrey M. Young, Vivek Sarkar, John Parrish, Nicole Wren, Tsz Hang Kiang, Akihiro Hayashi · MPLR 2023
• 18mA better way of doing Async RustPiotr Jastrzębski · Rustikon 2025
• 59mC++ on its way to exascale and beyondThomas Heller · Meeting C++ 2015
• 01h 07mAnother fundamental shift in Parallelism Paradigm?Michael S Wong · CppCon 2014
• 01h 06mTrue parallelism, with no concept of threadsAlfred Bratterud · Meeting C++ 2017