Arno Candel, Physicist & Hacker, 0xData
summary
Introduces a recurrent encoder that compresses a variable-length sentence into a fixed-length vector and a recurrent decoder that generates the target one word at a time using beam search; trains end-to-end on machine translation data (WM T/WMC benchmark cited) and matches a decade-plus engineered state of the art despite only a few researchers and ~one year of work. Claims generality across QA, speech and translation and publishes as sequence-to-sequence at NIPS.
AI-written summary · about
chapters (12)
- 0:00Distributed Deep Learning for Classification and Regression Problems using H20
- 0:39Outline
- 1:03H20: Open-Source (Apache v2) Predictive Analytics Plakform
- 1:15H20 Architecture - Designed for speed, scale, accuracy & ease of use
- 5:36H2O Deep Learning Architecture
- 6:24Application: Higgs Boson Classification
- 7:10Higgs: Derived features are important!
- 10:18MNIST: digits classification
- 11:05H2O Deep Learning beats MNIST
- 14:02ebay Text Classification
- 20:05Higgs Particle Detection with H20
- 20:32H2O Deep Learning Booklet R Vignette: Explains methods & parameters
related talks
16mThe End of Awkward AI Transcriptions
36mNeural Networks
01h 25mPaper Discussion: Attention is all you need
01h 00mReducing the Dimension of Language: A Spectral Perspective on Transformers
42mAttention and Memory in Deep Learning Networks
29mDetecting anomalous sequences using text processing methods
01h 02mAcoustic Word Embeddings for Low Resource Speech Processing
31mText and Code Embeddings