Rule #2: Double Tap. An Elasticsearch Journey of Resiliency

on YouTube ↗41m

" summary

Chronicle of specific fixes: Elasticsearch adds a translog write‑ahead log so uncommitted writes replicate to replicas, reducing data loss; deploys anti-affinity rules so shards don’t all land on one host; implements circuit breakers that abort queries before JVM OOMs; and replaces manual quorum settings with Zen 2 (released in 7.0) after using TLA+ model checking to find consensus bugs. These changes are implemented and materially reduce split‑brain incidents and recoverability failures.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 2:08What is Elasticsearch
3. 25:22How do you fix?
4. 30:36Properties
5. 35:10Full Specs

" related talks

• 16mTop Elasticsearch tipsNicolas Buisson · Devoxx Belgium 2017
• 44mSurviving ElasticsearchVitaliy Kharytonskiy · Highload fwdays'18
• 22mDesigning Concurrent Distributed Sequence Numbers ...Boaz Leskes · Berlin Buzzwords 2015
• 47mDatabase ResiliencyRavikumar Buragapu · Percona Community Live 2022
• 43mPartitions for Everyone!Kyle Kingsbury · Strange Loop 2013
• 46mTwo steps forward, one step backward - backward compatibility in ElasticsearchAlexander Reelsen · JCON 2020
• 41m(R)Evolution — You Know, for Search…Philipp Krenn · Highload fwdays'17
• 46mEdge Performance with In memory NoSQLLiviu Costea · Voxxed Days Vienna 2017