Easy Access to SOTA NLP Models with Ray and Hugging Face
summary
Hugging Face demonstrates running pretrained Transformers with the Tokenizers and Datasets libraries plus Ray Tune hyperparameter search to fine‑tune on GLUE MRPC. The demo loads MRPC quickly, runs batched tokenization, performs hyperparameter sweep over learning rate, batch size and seeds, and selects top runs; best result is MRPC (GLUE) — 0.86 accuracy, F1 0.90. Tokenizers encode ~1 GB in 20s and Wikipedia can load in ~9 GB RAM, and pruning stops bad trials early.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:28What is Transfer Learning?
- 1:56Sequential Transfer Learning
- 3:29Pretraining Transformers models BERT, GPT...
- 6:03Model: Adapting for target task
- 9:46A - Transfer Learning for text classification
- 10:55Fine-tuning is not always easy!
- 13:09Hugging Face: Democratizing NLP
- 14:51Transformers library: code example
- 15:16Transformers: model hub
- 15:42Tokenizers library
- 16:06Datasets library
- 17:09Datasets: code example
- 17:57Datasets: datasets hub
related talks
32mAn Introduction to Transfer Learning in NLP and HuggingFace
40mPanel discussion "NLP"
31mHugging Face ecosystem for Local AI/ ML
02h 10mIntro to Hugging Face: Fine-tuning BERT for NLP tasks
33mMachine Learning 2.0 with Hugging Face
19mBuilding Machine Learning Applications using Hugging Face
01h 09mReinventing Machine Learning with Transformers and Hugging Face
28mHuggingFace + Ray AIR Integration: A Python developer’s guide to