25 Years of Storage Research and Education: A Retrospective
summary
A career-long claim is that specific measurements plus simple system changes yield huge wins: a cluster-built sorter (NOWsort) hits ~6 GB/min versus the prior 1.6 GB/min record; whiskey key-value design separates keys from values on SSDs to cut read/write amplification and shows up to 100x speedups for large values; orthus non-hierarchical caching models show splitting/offloading beats classic caching when devices have similar latency; a user-space file-system semi-microkernel is implemented and outperforms ext4 at scale.
AI-written summary · about
chapters (39)
- 0:00Intro
- 0:43Outline
- 2:08Undergrad at Michigan: Al? Or not AI?
- 6:16My next choice: Computer Architecture
- 9:04First Few Projects
- 10:16A Class Project in Databases
- 12:33Project Proposal
- 13:06Result: NOW-Sort
- 14:49Main Lessons from Sorting
- 16:56Next Work: Search for Balance
- 19:14Graduation Dilemma
- 23:21River How to make sorting run fast
- 26:03Key: Measure Then Build
- 27:05Attack a Classic Problem: Costs of Layering
- 28:13An Idea: Gray Box Systems
- 29:45Refining Gray Boxes
- 31:05Examples
- 32:51Beyond Measurement
- 34:11In The Beginning
- 34:48Personal Drives
- 35:00Disruption: Solid-State
- 35:49Remainder of Talk
- 37:55LSM Background
- 38:23LSM Insert and Lookup
- 39:15Problem: 1/0 Amplification
- 40:12Solution: Wisckey
- 41:12Performance: Load
- 41:39Performance: Lookups
- 41:52Wisckey Summary
- 44:08Question: Can We Do Better Than Caching?
- 46:44Model: Results
- 47:42A Different Approach: Splitting (Offloading)
- 48:06Splitting: Results
- 48:52Caching vs. Splitting
- 50:05Classic Caching
- 50:16Non-Hierarchical Cachi
- 52:07Alternate Architectures
- 52:46Advantages
- 55:16Last Lesson: Thank People
related talks
49mKeynote Address: Everything You Always Wanted to Know about Storage Analysis...
30mCeph at 20 Years! Still the Best for Modern Storage
21mThe Resilient Hard Disk Drive (1984 to 2062)
42mHow shit works: the CPU
01h 00mMeasure, Then Build
45mA Brief History of Data Storage
52mHow shit works: Databases
14mFoundational HPC Systems for 2020 and Beyond