Airbyte: Open Source ELT For Everyone – Level Up With Custom Connectors

on YouTube ↗24m

" summary

Airbyte’s Connector Development Kit (CDK) demonstrates building a working source connector for the Exchange Rates Data API by running the generator, filling spec.yaml, connector definition, config/catalog and schema files, creating a Python venv and using Docker. The demo builds a usable connector in 20–25 minutes with zero Python beyond boilerplate, runs spec and check operations, reads one day of data, Docker builds and docker compose expose the connector in the UI and connection tests pass, and the connector can push to BigQuery S3 or Snowflake.

AI-written summary · about

" chapters (7)

1. 0:00Introduction and goals
2. 2:30Why Airbyte and custom connectors
3. 7:30Setting up the development environment
4. 13:13Building the connector definition
5. 16:25Testing and validating the connector
6. 18:35Deploying the connector locally
7. 20:21Q&A session

" related talks

• 37mAdrian Brudaru | Data Track | Compass AI & Data Summit 2025Adrian Brudaru · Compass AI & Data Summit 2025
• 28mRevolutionizing Financial Data Integration: The OpenBB EdgeI. Radovanovic · DSC Europe 23
• 40mThe Tools Empowering 5000 EmployeesJean-Mathieu Saponaro · Compass Tech Summit 2023
• 42mFrom zero to hero with Apache Kafka's connect APIRobin Moffatt · Berlin Buzzwords 2019
• 30mStream connector: Easily sending events and/or metrics from the Centreon open-source...DAVID BOUCHER · OSMC 2018
• 01h 18mIntroduction to orchestration ETL processes with Apache AirflowMiroslav Tomic · DSC Europe 24
• 34mAdvance Auto Parts and Open SourceJean-Georges Perrin, Adolfo Rodriguez · All Things Open 2020
• 28mHow Open Source Enables Innovation A Case Study with dbt and LightdashAmy Chen, Katie Hindson · OSA Con 2021