Big Data and SSDs
summary
Claims an open source ACT benchmark for mixed read/write real‑time loads and publishes GitHub code; measures drives over hours to spot pauses that appear after ~48 hours; reports examples like 150,000 read ops sustained with sub‑1ms latency on a high‑end Micron PCIe device versus ~6,000 IOPS for an Intel S3700 SATA drive; shows cost comparisons (10 TB replicated ~$250k on SSD vs ~$4M with RAM), warns about firmware variance, overprovisioning and RAID/controller bottlenecks.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 2:00Big Data Architecture
- 4:00InMemory Big Data
- 5:00What would you do with InMemory Big Data
- 6:50What is a specific example within the advertising community
- 7:32Why we should think about this
- 8:18SSDs
- 13:32Bottom line cost
- 17:13How to build a database optimized for SSDs
- 19:57Firmware for SSDs
- 20:58Measuring SSDs
- 22:30SSDs over time
- 25:54Overprovisioning
- 28:23Customers
- 30:40Other providers
- 32:08Our customers
- 36:32Test framework
- 41:18Data corruption
related talks
25mPerformance Analysis of NVMe SSDs and their Implication on Real World Databases
52mHow ZNS SSD's Are Changing the Landscape
14mOperational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field...
53mDatabase Hardware Selection Guidelines
12mFusionRAID: Achieving Consistent Low Latency for Commodity SSD Arrays
29mLarge-Scale Benchmark Results of NVMe Over Fabric Technology
53mOvercome Blast Radius Anxiety With Ultrafast Rebuilds and Performance for High-capacity SSDs
47mLoad Testing Storage Performance for SQL Server