Running Apache Spark on K8s: From AWS EMR to K8s
summary
Covers migrating Apache Spark workloads from AWS EMR to running Spark on Kubernetes. Describes operability benefits—reliability, portability, scalability—plus observability, efficiency and cost implications, based on a six-month real-world migration across three teams at Empathy.co to reduce cloud-specific dependencies.
AI-written summary · about
related talks
26mBuilding Apache Spark Clusters on K8s for AI Workloads
26mRe-architecting the Data Platform with Spark on Kubernetes
22mSPOK - Managing ML/Big Data Spark Workloads at scale on Kubernetes
21mSpark on Kubernetes: The Elastic Story
38mChallenges of and Solutions for Migrating Spark From Legacy Hadoop Clusters To Kubernetes
29mScaling Apache Spark on Kube to Apple Scale
28mCloud-Native Apache Spark: why & how to migrate your Spark
01mCloud-Native Apache Spark - Spark on Kubernetes Is Now Generally Available