Scraping Lyrics with Celery
summary
Demonstrates using the Python asynchronous task queue Celery to extract song lyrics from websites, showing how Celery works, how to choose a message broker and store results, and how to write Celery tasks. Includes practical coding examples using BeautifulSoup to scrape and inspect web page DOMs, discusses performance and result retrieval, and suggests improvements such as better storage and Celery primitives (chains, groups, chords).
AI-written summary · about
chapters (13)
- 0:00Welcome!
- 1:20About Celery
- 2:55How Celery works
- 3:36Message brokers
- 5:15Choosing a broker
- 6:35Storing results
- 9:30Coding with Celery
- 15:30Scrapping lyrics websites
- 16:46Inspecting the DOM
- 17:13Coding with BeautifulSoup
- 20:07Getting the results and performance
- 22:40Improvements: Better storage, Celery chains, groups, chords
- 24:15Wrapping up: Celery is as popular as Taylor Swift for distributed tasks
related talks
19mOptimizing Data Retrieval with Python Celery
24mDistributed Web Scraping in Python
24mimprov BOF How to Scrape Data from any Website
01h 01mWebcrawler
29mDive into Scrapy
29mHigh Fidelity Web Crawling in Python
45mPython tools for webscraping
34mScraping Techniques to Extract Advertisements from Web Pages