Building tiered data stores using Aesop to bridge SQL and NoSQL systems
summary
Describes Aesop, an open-source change-data-propagation system Flipkart built to bridge relational and NoSQL stores (MySQL, HBase, Redis, Kafka, ElasticSearch). Explains its partitioned, Zookeeper-coordinated architecture that provides timeline-ordered, at-least-once delivery for single-node and multi-node clusters, and gives deployment use cases such as payments, wishlists and streaming to analytics.
AI-written summary · about
chapters (12)
- 0:00Start
- 1:03What Data store?
- 4:25Scaling the data store (polyglot persistence)
- 7:55Caching/Serving Layer challenges
- 11:25Eventual Consistency
- 14:07Producer, Relay, Consumer
- 15:35Log Mining
- 17:33Data Consistency & Aesop
- 20:33Reliable delivery
- 25:13Event consumption
- 27:41Monitoring Console
- 28:47Summary
related talks
52mAesop Change Data Propagation : Bridging SQL and NoSQL Systems
32mBuilding a Real-time Ingestion Platform using Kafka and Aerospike
33mBuilding an Elastic, Resilient Data Layer to Power New Retail Experiences
57mMulti Tenant NoSQL and NewSQL Cloud data Design Patterns Deep dive
51mImmutable data stores and CORS for safety, flexibility and profit
29mFlipkart: Scaling Aerospike at Flipkart with Kubernetes
06mSnapdeal and Aerospike
49mHow Flipkart runs HBase across zones and clouds seamlessly