Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor, CalTech
summary
MXNet demonstrates a hierarchical parameter server that places a GPU as the level‑one server to cut GPU‑GPU vs CPU‑GPU bandwidth costs and records 91% efficiency at scale (88% at 256 GPUs) on K40/P3 clusters. Gluon hybridization generates symbolic graphs from imperative code to enable operator fusion and memory reuse. Replacing fully connected layers with tensor contraction and low‑rank tensor regression yields up to 65% model size reduction with negligible accuracy loss. Notebook demos use Tensorly + MXNet backend and show tensor kernels that approach single‑GEMM speed.
AI-written summary · about
chapters (10)
- 0:00Start
- 3:20Summary
- 6:13Inception V3 Network
- 10:41Data Parallelism
- 12:48How Mx Net Is Efficient at Scale
- 20:14How To Convert Imparative Core to Generating Symbolic Graphs in Gl
- 24:57How Do We Extend Matrix Operations to Tensors
- 27:09Tensor Contraction
- 39:34How Can You Detect Topics Automatically
- 40:45Conclusion
related talks
42mAlex Smola, Professor in the Machine Learning Department, Carnegie Mellon University
24mTianqi Chen, Computer Science PhD Student, University of Washington at MLconf Seattle 2017
20mApache MXNet, a highly memory efficient deep learning framework
30mApache MXNet 2.0: Towards Synergistic ML and DL with Standardization
39mAccelerate large-scale machine learning with NP on MXNet
29mAccelerate NumPy Data Science Workloads and Deep Learning Applications
01h 16mDeep Dive into Deep Learning with MXNet
38mThe new kid on the block: deep learning with GluonNLP