Exploring Azure Databricks Notebooks & Power BI for YugabyteDB
summary
Azure Databricks notebooks connect to YugabyteDB via JDBC and a Yugabyte Spark connector (com.yugabyte:ycql-spark-connector assembly), using YSQL on port 5433 and YCQL on 9042 across a three-node cluster. Demo imports convert CSV to Parquet then loads a sales_profit table in ~12 seconds, exports tables back to Parquet/Delta (Delta keeps versioning), and streams flight-delay CSV files into a YSQL table continuously (counts steadily increase, e.g. 63 records observed). Power BI uses ODBC/Postgres import (no DirectQuery).
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:12Azure Databricks + YugabyteDB
- 3:38YugabyteDB use cases
- 6:30Demo setup & YSQL
- 9:03Import YugabyteDB to Databricks
- 10:49Export Databricks to YugabyteDB
- 14:14Delta Lake & Parquet
- 16:43Databricks notebooks visualization
- 18:10YCQL connection & Cassandra
- 21:20Import/export with YCQL
- 24:08Import/export summary
- 25:34Real-time streaming setup
- 28:01Streaming flight delays
- 31:10Power BI + YugabyteDB
- 33:17Power BI demo & Q&A
- 38:01Closing & Q&A
related talks
43mAnalyzing Big Data in Azure cloud using Spark and Databricks
26mEfforts Toward Adoption and Uncovering Hidden Features
28mDisrupting the Status Quo with Distributed SQL to Deliver Data-Driven Solutions
52mPower BI + Azure Synapse Analytics. All you need to start creating a unified analytics solution
01h 27mYugabyteDB YSQL Development
01h 21mYugabyteDB YCQL Development
50mPower BI + Azure Synapse Analytics: Beginning your journey
30mBuilding Scalable Spring Applications with YugabyteDB