Towards new deep learning abstractions on top of exist frameworks

on YouTube ↗32m

" summary

Nervana Graph (ngraph) is a Python IR plus frontends and compiler backends that claim to let frameworks target CPUs, GPUs and Intel Crest without rewriting kernels. It exposes a Python API, axes named dimensions for static shape checks, automatic differentiation, operator fusion and memory-layout passes, and a C++ codegen using LLVM JIT proof of concept. Connectors target TensorFlow XLA and MXNet, and a heater transformer for MPI enables multi-node splitting and communication.

AI-written summary · about

" chapters (12)

1. 0:00Start
2. 1:01Description of Deep Learning
3. 1:56Background Questions
4. 3:59Tensorflow
5. 5:15Crest Family of Deep Learning Chips from Intel
6. 7:49Intermediate Representation of the Computational Graph
7. 10:59Onyx Format
8. 13:30Front End Connectors
9. 20:48Backends
10. 26:15Distributed Processes
11. 27:00Adding Heater Support for Mpi
12. 28:10Roadmap

" related talks

• 43mIntel Nervana Update + Productizing AI ResearchHanlin Tang, Naveen Rao · O'Reilly Artificial Intelligence Conference 2017 - New York, New York
• 19mNeuGraph: Parallel Deep Neural Network Computation on Large GraphsMing Wu, Lidong Zhou, Youshan Miao, Lingxiao Ma, Zhi Yang, Jilong Xue, Yafei Dai · USENIX ATC '19
• 16mPreview: Graph NNsAnand Iyer · OSDI '21 / USENIX ATC '21
• 30mAnyone can Build Great Deep Learning Applications - Deep NumpyCyrus Vahid · PyData Warsaw 2019
• 23mDeep dive in Nx BackendsPaulo Valente · Code BEAM America 2022
• 35mTowards Typesafe Deep Learning in ScalaTongfei Chen · Scale By The Bay 2018
• 25mNarayanan Sundaram, Research Scientist, Intel LabsNarayanan Sundaram · MLconf SF 2015
• 48mDynamic Neural Networks with DyNetYoav Goldberg · PyCon Israel 2017