Apache Arrow: In Pursuit of Speed

on YouTube ↗45m

" summary

Apache Arrow is presented as a columnar (vector) data format and toolkit for storing, processing and transmitting data without copying or marshalling. The talk explains how Arrow enables higher performance in data systems through Zero-Copy and No Marshalling, and examines engineering trade-offs and techniques developers use to achieve uncompromising speed.

AI-written summary · about

" chapters (8)

1. 0:00Introduction
2. 3:12Apache Arrow: what it is
3. 12:31In-memory data format
4. 16:46Zero-copy and marshalling
5. 22:51Zero-copy: slicing and transfer
6. 28:35Tools for speed
7. 36:53Summary and benefits
8. 37:56Q&A

" related talks

• 01h 02mApache Arrow: High-Performance Columnar Data FrameworkWes McKinney · Vaccination Database Talks (Second Dose) 2021
• 34mApache Arrow as a full stack data engineering solutionAlessandro Molina · PyCon Italia 2023
• 33mArrow in Flight - New Developments in Data ConnectivityDavid Li · OSA Con 2022
• 06mInterviewJacques Nadeau · Scale By The Bay 2019
• 39mTaming the language border (...)Uwe L. Korn · Berlin Buzzwords 2019
• 32mVectorized Query Processing for CPUs using Apache ArrowJacques Nadeau · Scale By The Bay 2019
• 46mEfficient Data formats for Analytics with Parquet and ArrowJulien Le Dem · Berlin Buzzwords 2016
• 22mAccelerating Analytics with Apache ArrowNeal Richardson · rstudio::conf 2019