Intake: Taking the Pain Out of Data Access
summary
Intake provides a single interface and YAML-backed catalogs that describe data sources (CSV, Avro, S3, Azure, HTTP) and return a pandas DataFrame (or array/xarray) with schema, metadata, quickplots and optional local caching. Catalogs can be shipped as Conda packages or hosted by an Intake server. Examples include a GitHub-issues driver and a US crime dataset (oddly high burglaries in 1918) and the plugin driver model makes adding new formats and credential templating straightforward and working.
AI-written summary · about
chapters (12)
- 0:00Introduction to Intake and its purpose
- 1:25Audience roles
- 3:10Traditional data access problems
- 5:19How Intake simplifies access
- 9:19Intake for all roles
- 11:24Intake catalogs benefits
- 15:13Custom drivers in Intake
- 18:09Publishing data with Intake
- 21:26Live demo
- 27:09Creating catalogs
- 29:57Intake ecosystem
- 32:05Summary
related talks
34mIntake - taking the pain out of data access
27mIngestify: Rethinking Ingestion for Complex Data
22mExploring the InfluxDB Ecosystem Integrations, Libraries, and Plugins
18mNo more sassy SaaS integrations
25mPandas on Jupyter: Improving Your Data Analysis Workflow
10mLightning Talk: We Built a Unified AI Asset Catalog in Backstage
20mData-as-a-product: A framework for collaborative data wrangling
05mAI Can't Use Data it Doesn't Have