Ursa Labs and Apache Arrow in 2019
summary
Ursa Labs forms as a nonprofit to fund Apache Arrow work with five full‑time engineers and a 2019 roadmap. Arrow defines a shared in‑memory tabular format used in 11 languages and ~1,000,000 installs/month. Spark integration with pandas UDFs yields up to 100x speedups even with conversions. Built components include the Flight transport, the Gandiva LLVM expression compiler, Parquet connectors and GPU (Rapids) support, all aimed at zero‑copy sharing and embedded query execution.
AI-written summary · about
chapters (18)
- 0:00Ursa Labs introduction
- 4:21Open source sustainability
- 9:48Legacy in data science
- 14:30Apache Arrow vision
- 20:15Arrow architecture
- 23:32Arrow benefits for pandas
- 28:36Arrow in Spark
- 30:20Arrow with RAPIDS
- 31:02Community feedback
- 32:51Growing Arrow ecosystem
- 33:57File formats and transport
- 35:45Gandiva project
- 38:15Arrow adoption
- 41:33Arrow and NumPy
- 42:36Null data handling
- 46:10Pandas rewrite process
- 49:47Python data classes
- 50:30Closing and Q&A
related talks
33mArrow in Flight - New Developments in Data Connectivity
42mState of the Apache Arrow Ecosystem:How your project can leverage Arrow!
32mThe expanding Apache Arrow universe
35mPractical Applications of Apache Arrow
34mLeveraging Apache Arrow for ML Workflows
22mAccelerating Analytics with Apache Arrow
29mFaster PySpark with Apache Arrow
27mApache Arrow - The columnar format! What's that?