Discover the Modern Way to Build Real-Time Analytics Solutions at Any Scale
summary
Shows a concrete before/after: a startup pipeline built from Postgres/Mongo/Elasticsearch + Kafka + Spark + S3 + Iceberg ends with data islands, heavy sync pain, and exploding ops costs. Replacing that with SingleStore yields single-platform ingestion (CDC from Mongo/Postgres, Kafka), Parquet/Iceberg + S3 history, and SQL that handles JSON, time series, vectors and key‑value. Vendors claim real customers doing 10M inserts/sec, achieving 50 ms SLA at millions of concurrent queries, and 50M vector comparisons/sec per core, with examples of big cost and latency reduction.
AI-written summary · about
chapters (10)
- 0:00Introduction and welcome
- 2:19Data world evolution overview
- 3:24Building a startup data pipeline
- 5:51PostgreSQL scaling challenges
- 9:54Adding real-time analytics components
- 14:35Revisiting architecture with NoSQL
- 17:48Introducing Kafka and data lakes
- 20:20Presenting SingleStore platform
- 28:30Simplifying architecture with SingleStore
- 30:42Summary and future trends
related talks
47mBuilding a Real-Time Analytics Database
33mThe rise and fall of the full stack database
32mBuilding a Real-time Ingestion Platform using Kafka and Aerospike
42mContemporary Approaches to Data at Scale
25mRemove Slow, Expensive ETL & Bring Data To The Data Lake For Lower Cost, More Performant Analytics
52mData storage for modern applications - Mike Lehan
32mBuilding Real-time Systems with Open Source Technologies
24mCracking the Cost Code: A Cost Optimized Approach to Distributed...