Web Scraping with BeautifulSoup & Yelp's API
summary
Scrapes YellowPages HTML with BeautifulSoup + requests, targets h2.class="n", div.phone phone-primary, div.street-address and div.locality, appends placeholders for missing values and zips lists into a pandas DataFrame (result: 3,032 entries scraped). Queries Yelp via the Yelp Fusion API using 50-item limits and offset paging, runs nested loops to collect multiple batches (offset bug appears then gets fixed), shares Google Colab notebooks at bit.ly/yp_scrape so the code runs end-to-end.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 4:45Scraping data uses
- 9:22Tools setup
- 12:13Scraping Yellow Pages
- 25:23Preparing the notebook and importing libraries
- 27:02URL list creation
- 33:25Extracting business data
- 49:54Data validation
- 51:46Switching to Yelp API
- 54:45Yelp API setup
- 1:01:43Yelp troubleshooting
- 1:03:31Final Yelp extraction & Q&A
related talks
28mData Harvest: Unlocking Insights with Python Web Scraping
36mThere will be Data: Scraping the Web with Python
38mBuilding mobile APIs with services at Yelp
24mimprov BOF How to Scrape Data from any Website
31mBeginning your own data engineering projects
34mWeb Scraping with Python
58mWeb Scraping with Beautiful Soup
02h 51mGathering data from the web using Python