JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative
summary
JANUS converts unmodified Python imperative DL code into optimized symbolic dataflow graphs by speculative graph generation after lightweight profiling, validating assumptions at runtime and falling back to imperative execution when needed. It handles dynamic control flow, types, and impure functions via added graph ops and a local Python heap copy to preserve semantics. Implemented on TensorFlow with CPython patches (~4,700 LOC + ~2,000 changed), it runs 11 models across 5 categories and achieves up to 47.6× speedup vs imperative and within 4% of symbolic performance.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:32Deep Learning (DL) Frameworks
- 0:59Two Paradigms
- 4:17Discrepancy between Python Programs and DL Graphs
- 8:53Challenge Summary
- 9:27Solution Speculative Graph Generation and Execution
- 10:44Overall Workflow on IANUS
- 12:58Additional System Aspects
- 14:31Evaluation Setup: Frameworks & Environments
- 15:13Evaluation Setup: Applications
- 15:26ImageNet Test Error with ResNet 50
- 16:46Model Convergence
- 17:18Normalized Training Throughout
- 20:20Conclusion
- 24:08JANUS Speedup over Imperative Execution: Breakdown
related talks
37mSymbolic Numeric Programming in Julia
01h 10mJAX: accelerated machine learning research via composable function transformations in Python
43mJulia: A Fresh Approach to Machine Learning
14mJAX: Accelerated Machine Learning Research via Composable Function Transformations in Python
29mAutomatic differentiation, a tale of two languages
11mIntro to JAX: Accelerating Machine Learning research
01h 28mA Tour of Julia
01h 27mIntroduction to Julia for Scientific Computing and Data Science