Service Dogs in the Cabin - Web Scraping Puppy Data with Microsoft Fabric - 2nd Leg
summary
Demonstrates web scraping inside Microsoft Fabric using PySpark notebooks (Python with httpx and selectolax plus geopandas and geopy for geocoding) and Power Query / M code, targeting the International Guide Dog Federation member pages for Europe and exporting structured JSON/CSV. Notebooks run in a configurable Fabric environment that requires adding public libraries and a gateway for browser requests; they return HTTP 200 and extract title, country, website, and detail links. Dataflows Gen2 works only if endpoints are hardcoded because Dataflows Gen2 does not support dynamic datasets; lazy-loading pages need JavaScript approaches.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:31Microsoft Fabric ingestion options
- 2:57Destination and source overview
- 4:45Setting up Fabric workspace
- 5:47Python environments for scraping
- 6:51PySpark scraping notebook
- 9:20Extracting data with CSS selectors
- 10:48Transforming scraped data
- 11:55Handling missing elements
- 13:22Power BI vs Data Flows Gen2
- 15:11Power BI dynamic data limits
- 16:57Data Flows Gen2 details
- 18:22Power BI report with scraped data
- 19:27Fabric scraping methods compared
- 20:30CSS selector resources
- 21:11Fundraiser and closing
related talks
50mMicrosoft Fabric Data Factory Q&A
27mModernize your data integration to enable petabyte scale analytics with Microsoft Fabric
40mIntroducing API for GraphQL and User Data Functions in Microsoft Fabric
49mBuilding an NFL Scouting Dashboard with Azure Data Services
49mExtend and enhance your analytics applications with Microsoft Fabric
46mMicrosoft Fabric Synapse data warehouse Q&A
50mFabric: What's new and what's next
47mData analytics for the era of AI