IOT, Timeseries and prediction with Android, Cassandra and Spark

on YouTube ↗49m

" summary

Collects Android accelerometer samples, posts them via REST, stores them in Cassandra using a per-user+date time-series partitioning with optional TTL, then streams recent rows into Spark Streaming through the DataStax connector. Slides 40,000 training samples, computes 15 features per window (mean, variance, stddev, avg abs diff, resultant, amplitude per axis), trains a Random Forest (10 trees) and reports ~92% accuracy versus 69% for a single decision tree. Live demo runs in realtime. Android accelerometer Cassandra time-series model Spark Streaming Random Forest (10 trees)

AI-written summary · about

" chapters (15)

1. 0:00Intro
2. 1:13IOT
3. 2:04Internet of Things
4. 3:35The Future
5. 6:53The Goal
6. 8:00Data Collection
7. 10:12Cassandra
8. 13:36Cassandra model
9. 17:11Spark core
10. 20:46Spark connector
11. 25:20Multiclass classification
12. 27:36Training model
13. 29:39Slice and analyze
14. 35:29Results
15. 41:13Conclusion

" related talks

• 33mAnalytics in the age of the Internet of Things #bbuzzLudwine Probst · Berlin Buzzwords 2015
• 38mReal-time analytics with Apache Cassandra & Apache SparkChristopher Batey · Berlin Buzzwords 2015
• 59mStreaming Big Data w/Spark, Spark Streaming, Kafka, Cassandra & AkkaHelena Edelson · Philly ETE 2015
• 52mSolving Analytical Problems using Apache SparkRohit Bhardwaj · Developer Summit 2020
• 20mBuilding realtime efficient queries in an IOT platformDinesh Narayanan · Data By the Bay 2016
• 21mIoT, Wavelets and Machine Learning in the Smart HomeKeita Broadwater · Scale By the Bay 2017
• 01h 36mSpark for Scala DevelopersBrian M Clapper · Scala World 2017
• 21mModeling Financial Data In Cassandra To Serve Real Time And Batch WorkloadsGokul Prabagaren · ApacheCon@Home 2021