Transformers for Game Log Data
summary
Applies a GPT-style decoder to game telemetry using PyTorch and nanoGPT-inspired code, trains on the Blade and Soul dataset (175M events, 77 base event types), experiments from tiny (~60k params) to mid models (~3–4M params) on Colab and A100 on GCP, uses vocabulary expansion to ~300 tokens and a context window of 24 (later 2048), gets usable player embeddings for clustering and best F1s ~0.86 via representation + random forest, but direct next-event generation performs worse.
AI-written summary · about
chapters (9)
- 0:00Transformers in game logs
- 5:03Why transformers for analytics
- 14:29Adapting logs for GPT
- 21:59Training on game data
- 31:36Evaluating predictions and segments
- 37:20Optimizing attention and context
- 38:45Production readiness?
- 40:07Key takeaways and future
- 41:32Q&A
related talks
01h 26mTransformers Inside Out (Parts 1 & 2)
28mGPT or how I learned to stop worrying and love the generative AI
01h 00mInside GPT – Large Language Models Demystified
47mInside GPT – Large Language Models Demystified
23mUnveiling the Journey of NLP
01h 40mA Deep Dive into NLP with PyTorch
01h 01mKeynote: Transformers: The Rise of ChatGPT
24mEfficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM