The Sparkling Solution that Scales to IoT
summary
Demo shows a 10-node EC2 cluster (Frankfurt) running Mesos/DCOS with three instances of a sleep app whose logs are tailed concurrently, and a UI/CLI that installs Cassandra, Kafka and Spark in minutes. Claim: DC/OS makes the data center look like one machine and enables bin‑packing that halved HubSpot’s AWS bill (160k → 80k) and cut Autodesk by ~60%. Kafka as durable log, Spark Streaming + SparkSQL write to Cassandra; Twitter ingest demo sometimes hits rate limits.
AI-written summary · about
chapters (10)
- 0:00Start
- 4:31TODAY'S LEGACY DATACENTER
- 5:50INSTALLING AN APPLICATION WITH STATIC PARTITIONING
- 6:07RESIZING AN APPLICATION WITH STATIC PARTITIONING
- 16:28MESOS - OVERVIEW
- 18:18DATACENTER OPERATING SYSTEM
- 20:27DCOS STACK
- 24:27DCOS NODE TYPES
- 33:48INSTALL CASSANDRA
- 34:04CASSANDRA FRAMEWORK
related talks
41mApplying Geospatial Analytics at a Massive Scale using Kafka, Spark and Elasticsearch on DC/OS
12mEdward Hsu, Mesosphere | DockerCon 2018
19mKeynote Panel Discussion: SMACK in the Enterprise
52mSolving Analytical Problems using Apache Spark
30mArchitecting Applications With Multiple Open-Source Big Data Technologies
39mDeploying Fast Data Pipelines
54mHarnessing the Power of Spark & Cassandra within your Spring App
42mStreaming Data Pipelines on Apache Mesos: Lessons Learned