Designing your Data: The Bread & Butter of APL Performance
summary
Practical techniques encode domains into arrays by pairing a relational model with the array model: use inverted tables (column stores) so each field uses compact native types, attach implicit indexes and preallocated buffers for amortized append, and apply symbol interning to shrink strings (example: 400 bits → 240 bits, ≈40% smaller in the worked example). Replace 64-bit pointers with pointer vectors constrained to 8/16/32-bit ranges, use bitmask selection and key-based permutations to vectorize filters, stable reorderings and fast lookups instead of per-node traversals.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 0:34Background
- 1:09What is APL
- 1:40Problems with APL
- 2:44Experts work on APL
- 3:18Designing your data
- 3:53The relational model
- 5:21The array model
- 6:31The easy tactic
- 8:44Aggregation
- 10:59Inverted Tables
- 13:13Implicit Data Structures
- 16:59Symbol Tables
- 20:27Pointer vectors
- 23:58Aggregate tagging
- 26:34Representations
- 30:15Boolean Masks
- 34:46Array Ordering
related talks
48mDesign of data-intensive applications
42mAdventures in Efficiency and Performance
56mWhy Data Access Patterns Have Always Mattered
51mDesigning for Performance
52mProgramming Obesity: A Code Health Epidemic
36mStatic Analysis of APL for Tooling and Compliance
27mHigh Performance Architecture
40mArchitecting for Data-Intensive Workloads