LODIC: Logical Distributed Counting for Scalable File Access
summary
A Linux kernel patch implements a per-process reference counter that embeds small local counters into unused bits of x86 PTEs and locates them by scanning a process’s VMA list near the stack, avoiding per-core allocation. The system uses per-process local counters, process-space reverse mapping, and counter embedding, is evaluated on RocksDB and NGINX, and reports a 65× throughput increase for shared file-block reads versus stock Linux, 2.5× speedup for NGINX under 50 clients, and 13× lower memory pressure versus per-core distributed counters.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:22Background
- 1:15Reference counter in the kernel object
- 1:35Manycore scalability in kernel object access
- 2:35Issues in distributed reference counter
- 3:04Existing works for per-core reference counter
- 4:28Characteristics of kernel objects
- 5:40Cause for counter contention
- 6:14LODIC : Logical Distributed Counter
- 7:28Objective
- 7:50Key techniques
- 8:22File Mapping
- 8:49Find VMA of mapped page
- 9:07Reverse Mapping
- 11:05Counter Embedding
- 12:17Throughput on shared file block read
- 12:47Web server throughput
- 13:22Counter query latency
- 13:50Conclusion
related talks
15mNrOS: Effective Replication and Sharing in an Operating System
23mBreaking the limits of x86
40mCentralized Counters in a Distributed World Event Based Realtime Statistics using eTally
17mSingularFS: A Billion-Scale Distributed File System Using a Single Metadata Server
29mAccelerating Filesystem Checking and Repair with pFSCK
21mNew Techniques to Improve Small I/O Workloads in Distributed File Systems
16mDisaggregation and the Application
21mDistCache: Provable Load Balancing for Large-Scale Storage Systems with Distributed Caching