To Spark or Not to Spark
summary
If you need joins, cross-source aggregation or a JDBC endpoint over Cassandra, Spark plus the Cassandra connector gives that: use spark thrift server for a JDBC surface, DataFrame SQL for optimized distributed aggregates, or structured streaming (micro-batches) for windowed Kafka work. Tradeoffs: new cluster/operator costs, loss of some DB-specific types, and poor fit for single-machine jobs. Alternatives include Flink/Presto, Kafka Connect or the DataStax bulk loader and cqlsh copy for simpler ETL.
AI-written summary · about
chapters (14)
- 0:00Introduction to Spark
- 1:06Spark use cases overview
- 3:14Spark for SQL and analytics
- 8:55Benefits of Spark for analytics
- 11:02Drawbacks of Spark
- 12:07Alternatives to Spark
- 16:26Spark for ETL
- 19:41ETL benefits and drawbacks
- 22:31Spark for streaming
- 24:39Streaming API comparison
- 26:23When to use Spark for streaming
- 27:06Pros and cons summary
- 29:11Spark and Cassandra deployment
- 32:15Q&A
related talks
54mHarnessing the Power of Spark & Cassandra within your Spring App
40mSpark Analytics on Cassandra Data
52mSolving Analytical Problems using Apache Spark
38mReal-time analytics with Apache Cassandra & Apache Spark
06mInterview
55mUse Spark from Anywhere
59mStreaming Big Data w/Spark, Spark Streaming, Kafka, Cassandra & Akka
30mArchitecting Applications With Multiple Open-Source Big Data Technologies