SOWalker: An I/O-Optimized Out-of-CoreGraph Processing System for Second-Order...
summary
SOWalker is an out‑of‑core implementation that cuts block I/O to 6–41% of GraphWalker and 8–73% of GraSorw and achieves up to 10.2× speedup over GraphWalker (up to 5.7× over GraSorw) by combining a walk matrix that tracks previous/current‑block walk counts, a benefit‑aware I/O model solved with simulated annealing to pick block sets, and a block‑set oriented walk updater plus 128‑bit compact walk records; benchmarks on Twitter and UK‑Union show much higher I/O utilization and update rates.
AI-written summary · about
chapters (9)
- 0:00SOWalker intro and second-order walks
- 2:15Out-of-core graph processing challenges
- 3:40Limitations in second-order walk systems
- 6:56Walk Matrix for block states
- 8:21IO model for optimal block scheduling
- 11:17Block set work updating
- 13:01Experimental results and performance
- 15:12Evaluating optimizations
- 17:00Conclusion
related talks
20mGraphWalker: An I/O-Efficient and Resource-Friendly Graph Analytic System...
32mLarge-Scale Graph Processing on Emerging Storage Devices
21mSeraph: Towards Scalable and Efficient Fully-external Graph Computation via On-demand...
19mOasis: An Out-of-core Approximate Graph System via All-Distances Sketches
25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed...
09mEfficient Large Graph Processing with Chunk-Based Graph Representation Model
01h 06mBest practices when accessing Big Data or any other data!
25mUsing Set Cover to Optimize a Large-Scale Low Latency Distributed Graph
