Auto-encoders in the wild... of telco land
summary
Implements MXNet symbolic autoencoders with a 200-dimensional embedding and trains with cosine loss + transpose reconstruction while adding a softmax task to predict the single rarest action. Adds a sparsity penalty to shape hidden vectors, finds batch-size sensitivity (transpose task needs large diverse batches), and extends to LSTM sequence autoencoders plus KL-style shaping. Results: rare signals preserve much better and downstream classifiers improve; GPU memory limits training speed.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 0:28Talk overview
- 1:12Neural networks and MXNet
- 6:34Telecom data problem
- 9:41Matrix factorization attempt
- 10:43Auto-encoders explained
- 12:32Building in MXNet
- 15:00MXNet symbolic API
- 16:46Training the model
- 17:28Loss functions
- 18:54Sparse embedding challenges
- 19:57Transpose loss for rare actions
- 21:01Enforcing sparsity
- 22:26Multi-task learning
- 23:29Sequence modeling
- 24:33Variational auto-encoders and privacy
- 25:15Summary
- 25:45Q&A: Architecture
- 27:10Q&A: Pre-processing
- 27:42Q&A: Loss and batch size
- 28:33Q&A: Training time and GPU
- 29:20Q&A: Dataset understanding
- 30:04Q&A: Privacy in embeddings
related talks
17mProcedural Modeling Using Autoencoder Networks
56mRandom Embeddings, Matrix-valued Kernels and Deep Learning
40mAn Occam's Razor Principle for Transformers?
52mPredictive Coding Models of Perception
39mA Learning Story: Love, Scala & Artificial Intelligence
40mYou can AI like an Expert
45mAnomaly detection with TensorFlow
12mEmergent Trends in Large Multimodal Models