AWS On Air ft. Amazon EMR Serverless Quick Tip 1

on YouTube ↗05m

" summary

Demonstrates Amazon EMR Serverless (preview) running a PySpark job on EMR 6.5 that reads NOAA Global Summary of the Day from S3, auto-starts and auto-stops, and dynamically scales executors (observed 20 concurrent tasks) until the job succeeds. Shows using an S3 log URI and a Spark history server via Docker to inspect event logs and executors, and extracts ~4 million weather readings for 2020. Uses standard EMR Serverless APIs and S3 for logs.

AI-written summary · about

" chapters (4)

1. 0:00Introduction
2. 2:32Spark History
3. 3:25Job History
4. 4:36Conclusion

" related talks

• 41mEMR ServerlessRadhika Ravirala · AWS Summit San Francisco 2022
• 28mInnovations in AWS analytics: Data processingChris Horder, Allison Quinn · AWS Summit Sydney 2025
• 31mProductizing Machine Learning over Big Data with AWS toolsMaximo Gurmendez · Global Artificial Intelligence Conference 2020
• 38mUnleashing the serverless data pipelinesSunita Koppar · Data Con LA 2021
• 48mAccelerate science, research, and innovation with AWSJason Duffe, Sanaz Vafaei, Arthur Krughkov · AWS Public Sector Symposium Ottawa 2023
• 32mMove to managed analyticsDamon Cortesi · AWS AMER Summit May 2021
• 47mAccelerate Your Data & Analytics Projects with Amazon EMR and AthenaStephen Roux, Weibo Gu · AWS Summit Paris 2024
• 27mVisualizing Cloud SystemsLynn Langit · GOTO Chicago 2020