~/talk

On the Feasibility of Parser-based Log Compression in Large-Scale Cloud Systems

on YouTube 12m
Talkat FAST '21 · 2021-02-23..02-25· [databases]

speakers

name:Junyu Wei
links:none on file
name:Guangyan Zhang
links:none on file
name:Yang Wang
links:none on file
name:Zhiwei Liu
links:none on file
name:Zhanyang Zhu
links:none on file
name:Junchao Chen
links:none on file
name:Tingtao Sun
links:none on file
name:Qi Zhou
links:none on file

resources

video:https://youtu.be/3V45WpSmUnM
slides:none on file

summary

LogReducer is a C++ reimplementation of parser-based log compression that targets numerical fields and fixes logzip bottlenecks, using delta timestamps, correlation identification, and elastic encoding. On 1.76 TB of production logs it compresses to 34.25 GB, achieving up to 4.8× better ratio than LZMA and up to 4× better than logzip, while running two orders of magnitude faster than logzip and with throughput comparable to LZMA.

AI-written summary · about

chapters (13)

  1. 0:00Introduction
  2. 0:36Logs
  3. 1:35Compression
  4. 3:50Design
  5. 4:45Engineering
  6. 5:26Comparison
  7. 5:38Compression Techniques
  8. 6:19Stacking Technique
  9. 7:19Elastic Encoding
  10. 8:27Reverse Encoding
  11. 8:38Architecture
  12. 9:35Evaluation
  13. 10:58Conclusion

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.