Implementing a highly scalable Stock prediction system with R, GemFire and Spring XD
summary
Realtime pipeline that trains a linear regression on historical stock ticks and scores the next-second moving median using R + Spark MLlib, Spring XD streams and Apache Geode (GemFire) as the in‑memory store. Demo ingests Yahoo feed (or a simulator replaying ~41,000 points, ~36,000 indicator rows), stores raw/enriched/prediction regions, and pushes updates to a JavaFX dashboard. Typical prediction error runs ~1.3–1.8%, streams and spark jobs are launched via Spring XD DSL and spark-submit.
AI-written summary · about
chapters (9)
- 0:00Introduction and overview
- 1:17Stock prediction system goals
- 10:06Machine learning fundamentals
- 21:34GemFire architecture basics
- 33:47Spring XD data pipelines
- 37:42Spark MLlib and RDDs
- 48:03Live system setup and demo
- 1:08:10Demo walkthrough and code
- 1:19:53Q&A session
related talks
11mScaling R with Spark
37mBig Data Analytics, Machine Learning, Real Time Processing
30mPredictive trading
33mReal-time Trade Data Processing and Machine Learning at Giga-Sca...
30mSmart Streams: A Real-time framework for scoring Big and Fast Data
45mReal-time, Streaming Advanced Analytics....
53mReal-Time Big Data Analytics with R
28mEfficient Time-Series Forecasting with Thousands of Local Models on Databricks