FoundationDB Record Layer: Open Source Structured Storage on FoundationDB
Talkat FoundationDB Summit 2019 · on Monday, 2019-11-18· at 15:40· Grand Ballroom 5, Marriott Marquis San Diego Marina·
summary
Apache-licensed library that implements a relational-like layer on FoundationDB using protocol buffers and transactionally maintained indexes, with an asynchronous pipelined query executor and a simple planner. It handles schema migrations, complex indexes (aggregates, rank, full‑text), and the hard part of online index build logic (indexing codebase >2,000 lines to handle batching, conflicts and FDB limits). Apple runs it in CloudKit at massive scale — billions of record stores with some stores at terabyte size — and the system is proven to work in production.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:27Schema and storage format
- 2:11Rich set of transactional indexes
- 3:05Query optimization and execution
- 4:52Building a database is hard!
- 6:24Layer engineering is still hard!
- 8:08Building new indexes
- 10:36Asynchronous query execution
- 12:40Ready for multi-tenant workloads
- 15:31Deep extensibility for building "higher layers"
- 16:37Community direction
related talks
25mFoundationDB Document Layer
46mFoundationDB: The Best-Kept Secret of New Distributed Architectures!
22mSolving Everyday Data Problems with FoundationDB
34mFoundationDB, the Black Knight
50mBuilding the Next-Generation Backend on FoundationDB
37mTechnical Overview of FoundationDB
22mHow FoundationDB powers SnowflakeDB's metadata!
17mState of the FoundationDB Community