HPC application in analysis of large scale complex networks

on YouTube ↗01h 17m

" summary

Shows how Networkit on a GPU (tested on Google Colab T4) loads a ~YouTube dataset of about 1.1M nodes and ~3M edges, computes degree distributions (power‑law exponent ≈2), reports max degree ~28,754 and global clustering ≈0.08, and skips full diameter because APSP is costly. Implements degree‑neighbor and clustering‑vs‑degree helpers, runs PLM (Louvain) and label‑propagation community detectors (≈4.4k vs ≈25k communities), compares partitions with mutual information (~0.84 between Louvain runs), and demonstrates that Networkit gives fast, parallel C++ backend access from Python for large graphs.

AI-written summary · about

" chapters (10)

1. 0:00HPC in network analysis
2. 3:17NetworkX limitations
3. 14:04Complex network properties
4. 22:13Clustering coefficient
5. 30:16Network distance and small-world
6. 37:50Assortativity index
7. 41:01YouTube friendship case study
8. 59:27Community detection
9. 1:06:40Community partitioning algorithms
10. 1:15:55Future directions and NetworkKit

" related talks

• 39mThorny path to the Large Scale Graph ProcessingAleksey Zinovev · HighLoad++ 2014
• 50mBig Data on Smallish MachinesGuy Blelloch · Parallel and Distributed Algorithms for Inference and Optimization (Simons Institute, Oct. 21–24, 2013)
• 16mAccelerating Distributed Graph Learning by Using Collaborative In-Network...Kai Chen, Junxue Zhang, Zhaoyi Li, Jiawei Huang, Yijun Li, Jingling Liu, Hui Li, Xiaojun Zhu, Shengwen Zhou, Jing Shao, Xiaojuan Lu, Qichen Su, Jianxin Wang, Chee Wei Tan, Yong Cui · USENIX ATC '25
• 26mLarge Scale Graph Processing & Machine Learning AlgorithmsVenkatesh Ramanathan · MLconf Atlanta 2017
• 39mThorny path to the Large Scale Graph ProcessingAleksey Zinovev · HighLoad++ 2014
• 14mP3: Distributed Deep Graph Learning at ScaleSwapnil Gandhi, Anand Padmanabha Iyer · 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21)
• 44mHow We Store and Analyze a Large Social GraphMaksim Bartenev · HighLoad++ 2014
• 30mScalable Graph-Parallel Inference Algorithms and SystemsJoseph E. Gonzalez · Parallel and Distributed Algorithms for Inference and Optimization (Simons Institute, Oct. 21–24, 2013)