Rearchitecting CouchDB Secondary Indexes on Top of FoundationDB

on YouTube ↗18m

" summary

Rewrites CouchDB secondary indexes to run on FoundationDB by making Mango indexes update in the same document transaction (so queries after a write see the index up to that write), storing index metadata and build sequence in FDB and using changefeed-based background builds, encoding CouchDB collation via tuples plus ICU sort strings, implementing MapReduce views with an FDB-backed global job queue and an ID index to track per-document keys, replacing on-disk Lucene storage with a Lucene directory layer on FoundationDB served by Java in-memory nodes; build latency increases but query latency drops significantly.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 1:13Mango
3. 5:40MapReduce
4. 8:51Building
5. 12:10Reduce
6. 14:52Text Search
7. 16:41Current Status

" related talks

• 22mSolving Everyday Data Problems with FoundationDBRyan Worl · FoundationDB Summit 2018
• 37mTechnical Overview of FoundationDBEvan Tschannen · FoundationDB Summit 2018
• 25mFoundationDB Document LayerBhaskar Muppana · FoundationDB Summit 2018
• 50mBuilding the Next-Generation Backend on FoundationDBOleg Illarionov · HighLoad++ Moscow 2018
• 46mFoundationDB: The Best-Kept Secret of New Distributed Architectures!S.LE ROUX, P.ZEMB · Devoxx France 2023
• 01h 02mNext Generation Data Storage with CouchDBJan Lehnardt · MountainWest RubyConf 2008
• 26mFuture of FoundationDB Storage EnginesSteve Atherton · FoundationDB Summit 2018
• 22mHow FoundationDB powers SnowflakeDB's metadata!Ashish Motivala · FoundationDB Summit 2018