Retrofitting High Availability Mechanism to Tame Hybrid Transaction/Analytical Processing
Talkat 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21) · 2021-07-14..07-16·
summary
Vegito retrofits replica-based HA so backups serve analytical queries while primaries handle OLTP, using a gossip-style epoch oracle to give consistent, millisecond-granularity visibility, a block-based multi-version column store with row-split/column-merge optimizations for fast scans, and two-phase concurrent index updates (buffered leaf inserts with batched rebalances) to keep writes parallel. It reports 1.9M TPC-C NewOrder tx/s + 24 TPC-H-equivalent queries/s with <20 ms freshness, ~5% OLTP slowdown, and recovery under 60 ms.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:27Typical workloads of databases
- 1:14Requirements Performance
- 1:53Data analysis: TP+ETL+AP
- 2:40HTAP alternatives
- 3:29VEGITO a distributed in-memory HTAP system
- 4:35Effects of VEGITO
- 5:00Log cleaning Log shipping TP threads append logs to queue ► Cleaners drain logs
- 5:56Epoch-based design
- 7:04Consistent epoch assign Gossip-style epoch assign
- 8:03Parallel log cleaning Clean logs matching TX dependence
- 8:36MVCS Multi-version column store (MVCS)
- 9:21VEGITO: block-based MVCS
- 9:59tree-based index
- 10:25Two-phase concurrent updating
- 10:56Fault tolerance
- 11:36Evaluation Setup
- 12:33HTAP workloads
- 12:55Recovery
- 13:09Conclusion
related talks
01h 01mHTAP Workloads: Challenges and Solutions
10mBuilding High-Performance Financial Data Pipelines
49mHTAP with Azure Cosmos DB: Hybrid Transaction & Analytical Processing
17mMako: Speculative Distributed Transactions with Geo-Replication
29mHTAP vs OLTP/OLAP
41mTransactions: myths, surprises and opportunities
29mOracle TimesTen: extreme performance for latency-critical applications
46mDeploying Maximum HA Architecture With Postgres