Slimming down the journal
summary
Binary journal format changes reduce disk use by changing offsets to 32-bit offsets (saves ~500 MB but limits files to 4 GB), storing field offsets in a shared prefix tree called compact mode (saves ~800 MB at cost of an extra indirection and possible slower iteration), and letting uncommon fields skip indexing via configurable indexing (saves ~400 MB). Storing unique small fields inline saves ~200 MB more; trade-offs include lost instant indexing, rewrite-on-archive complexity, and extra runtime lookup work.
AI-written summary · about
chapters (10)
- 0:00Introduction to systemd Journal
- 2:04Journal binary format overview
- 7:01Problems with current format
- 8:4932-bit offset optimization
- 10:16Compact mode with trees
- 13:47Caching entry array objects
- 14:49Configurable field indexing
- 16:35Field deduplication tweaks
- 18:26Future compression ideas
- 21:13Optimizations summary and Q&A
related talks
20mA Powerful Logs Management Solution We All Have and Use but...
27mEstimating dm-vdo storage savings
35mOptimizing sysfs and procfs
20mOptimizing Checkpoints with Built-in Memory Page Compression
35mRethinking Linux tools with Compact Debuginfo Formats
12mEvery Mapping Counts in Large Amounts: Folio Accounting
54mWhat Lies Beneath
14mDecoupling Large Folios from Transparent Huge Pages