The Future of the Past: Challenges in Archival Storage

on YouTube ↗01h 19m

" summary

Argues that preserving bits and their meaning for centuries requires choosing media by concrete tradeoffs: tape (5–100 s load, ~200+ MB/s, ~$5/TB) and optical (≈180 MB/s, tens $/TB) are proven; disk (~10 ms, ~$20/TB) has short lifetime (5–7 years); flash is fast (≈50 μs, ~1 GB/s) but costly (~$80/TB) and leaks without refresh; Project Silica glass promises millennia durability with ~50 ms reads and ~50 MB/s, while DNA: day-long reads is ultra‑dense but currently very expensive. Describes built systems (MAID, Pergamum, Pelican, F4) and concrete integrity/search solutions like secret-sharding and per-media indexes.

AI-written summary · about

" chapters (27)

1. 0:00Intro
2. 0:39Why archival storage?
3. 4:26Why do we care about archival storage
4. 6:51Preserving information over the millennia
5. 8:38Media reliability
6. 10:14Data integrity
7. 14:08Preserving meaning over time
8. 18:41Media: performance
9. 20:13Preserving the bits: media
10. 21:08Media: tape
11. 24:12Media: disk
12. 26:07Media: glass (Project Silica) Microsoft-led project to write data in glas • Performance
13. 28:44Media: comparison
14. 30:16Media: futures
15. 31:51Preserving the bits: long-term retention Several key questions about media? - Do bits last forever after they're written? - How likely are bits to "rot"? How much maintenance do they require? Some media are relatively "hands off"
16. 32:46So what's the best option?
17. 33:28The economics of archival storage
18. 35:36What happens when storage stops getting cheaper over time?
19. 36:32Consider cost over time
20. 37:39Reliability: "blast radius"
21. 39:01Preserving the bits: integrity & security
22. 43:13Archival storage projects
23. 46:19Preserving the meaning of data
24. 48:21Managing information: search
25. 51:14Summary: challenges in preserving data for the long- term
26. 52:09Conclusions
27. 54:02Acknowledgments

" related talks

• 57mA Brief History of Data StorageEli Holderness · NDC Sydney 2024
• 45mA Brief History of Data StorageEli Holderness · Build Stuff 2024
• 01h 00mA Brief History of Data StorageEli Holderness · NDC Oslo 2023
• 56mA Brief History of Data StorageEli Holderness · NDC Porto 2023
• 01h 05mA Brief History of Data StorageEli Holderness · NDC London 2024
• 01h 00mA Brief History of Data StorageEli Holderness · NDC Copenhagen 2026
• 41mWhy would you store digital information in DNA?Robert Grass · Øredev 2025
• 34mData @Scale Seattle- Aaron OgusAaron Ogus · Data @Scale Seattle 2015