Empower Your Projects with Prefect's Pipeline Magic
summary
Hands-on workshop shows Prefect 2.19 pipelines that fetch tweets, publish to Upstash Kafka, run spaCy sentiment, and persist results to MongoDB Atlas while using a Postgres backend and Docker for local services. Code lives on the GitHub repo; flows, tasks, retries and logging are real and deployed as scheduled runs (one demo uses a 3s interval). The demo copes with a flaky API (≈25% 404s) via retries, uses subflows and logging, and produces live records (counts rise as runs complete).
AI-written summary · about
chapters (12)
- 0:00Conference overview
- 2:22What is Prefect
- 9:07Notebooks vs production pipelines
- 11:39Prefect workflows
- 14:29Setting up the local development environment
- 23:45First Prefect flow
- 37:12Logging and retries
- 39:40Sentiment analysis pipeline
- 55:53Kafka streaming
- 1:03:04MongoDB storage
- 1:16:26Scheduled flow deployment
- 1:22:50Live demo
related talks
24mUncovering Community and Project Insights Through Data Driven Methods
01h 07mBuilding Reactive Pipelines
30mDeploy end to end ML pipeline using Apache spark ...
01h 25mBuild and Deploy an End-to-end Streaming NLP Insight System
44mShaping the Future of Real-time Data Pipeline
36mHuman-Friendly, Production-Ready Data Science Stack with Metaflow & Kubernetes
01h 47mRadanalytics.io streaming applications workshop
49mOrchestrating data and ML workflows with Apache Airflow