The Fletcher Framework: Bringing Apache Arrow to FPGA
summary
Fletcher brings Apache Arrow in-memory tables directly to FPGA by generating hardware interfaces from Arrow schemas, producing HDL templates and simulation tops and providing Python/C++ runtimes plus platform plugs for OpenPOWER CAPI SNAP and AWS F1. It claims zero-copy integration with over 11 languages, shows FPGA regex kernels running in milliseconds while CPU/Python/Java take seconds, and measures about 10 GB/s CAPI 2.0 throughput for Arrow string reads/writes.
AI-written summary · about
chapters (28)
- 0:00Intro
- 1:09Outline
- 1:48Contemporary Big Data systems
- 3:30A string
- 6:58Relative impact on accelerators
- 9:13Sneak preview / motivation: Regular Expression Matching
- 9:57Regex on 1 GiB of tweet-like strings
- 11:49Overcoming serialization bottlenecks
- 13:19Apache Arrows
- 15:08Arrow in- memory dataset, basic example
- 17:04Integrating FPGA and Arrow
- 18:10Example: Interface for accelerator parsing strings Fletcher
- 21:41Generated interface overview - Architecture based on library with streaming primitives
- 25:48Combining Buffer Readers into ArrayReaders
- 27:15Verification
- 28:03High-level architecture generation: Fletchgen
- 29:22Fletcher run-time stack
- 31:04Mini-tutorial: Fletcher "Hello, World!"
- 32:01Create an Arrow Schema
- 33:09Create a Record Batch
- 34:03Generate the design
- 35:22Implement the kernel
- 36:17Simulate the design
- 38:10Write host-side software
- 39:27Target a Platform
- 41:10Spin-off projects & future work
- 43:47Summary
- 44:40References
related talks
01h 02mApache Arrow: High-Performance Columnar Data Framework
33mExtending Pandas using Apache Arrow and Numba
32mVectorized Query Processing for CPUs using Apache Arrow
23mSparrow, Pirates of the Apache Arrow
45mAccelerating OpenSearch With Streaming: Apache Arrow, Flight, Data...
06mInterview
26mUnlocking Scalable Distributed Training With Arrow Data Cache on Kubernetes
02h 50mFunctional Array Funhouse Intensive - Part 3 of 5