Smokey and the Multi-Armed Bandit featuring BERT Reynolds
summary
Fine-tuned two BERT variants (TensorFlow and PyTorch, using the Transformers library from Hugging Face) on the Amazon Customer Reviews dataset and deploys them as SageMaker endpoints, then trains a contextual multi-armed bandit (Vowpal Wabbit in SageMaker) to shift live traffic via Kinesis/S3/DynamoDB. Short training yields ~40–42% accuracy (8 minutes) and near 50/50 split in decisions, with DistilBERT noted as 97% of accuracy and one TensorFlow endpoint briefly failing during the run.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 0:49BERT and reinforcement learning
- 5:02Traditional A/B tests
- 7:30Multi-armed bandits
- 9:54BERT variants and fine-tuning
- 11:25Reinforcement learning setup
- 12:29System workflow
- 17:04Live demo
- 23:25SageMaker code walkthrough
- 28:19Continuous deployment
- 31:28Results and analysis
- 35:20Resources and conclusion
related talks
45mBuild NLP Models with Amazon SageMaker
27mEnhancing Personalization Using Recommendation Systems & Multi-Armed Bandits
33mHow to build personalization models using Amazon SageMaker
30mBuild consistent and portable ML environments with containers
25mAccelerate NLP training with Amazon SageMaker
21mLeveraging Multi-Armed Bandit Algorithms for Dynamic Decision Making
51mProfundizando en Aprendizaje Automático para Desarrolladores
31mTrain and tune ML models to the highest accuracy using Amazon SageMaker