Behind the Streams: In Depth Look At ML&AI Powered Analysis of FPS Gameplay
summary
A pipeline that downloads 8+ hour streams, cuts them into raids, detects the map (Reserve, Customs) and extracts kill counts and action timestamps using Tesseract OCR, frame grabs via ffmpeg, and audio labels from a Hugging Face model. It runs frame sampling every 15s (quarter‑second when needed), takes ~30–40 minutes to process an 8‑hour file, stores results in SQLite and a Blazor web UI, and returns useful but imperfect raid clips (examples: an 18‑minute Customs raid with 3 PMC kills; rare runs show 8+ kills).
AI-written summary · about
chapters (8)
- 0:00Introduction and speaker background
- 8:09What is Escape from Tarkov
- 15:07Project motivation and goals
- 21:43Technologies and tools used
- 31:35OCR and image processing for data extraction
- 37:53Web application and results
- 45:41Challenges and future improvements
- 48:30Conclusion and Q&A
related talks
51mVideo Analytics for Football games
35mDecoding bias and narrative in competitive video games
17mWhat Moves Players? Visual Data Exploration of Twitter and Gameplay Data
36mVideo analytics at scale: Deep Learning, Computer Vision, ML & Spark
50mThe tech behind modern AI/ML use cases in Gaming
20mEngaging Bots for Multiplayer Games
53mVideo Analytics for mere mortals with NVIDIA DeepStream SDK
41mEnhancing Event Analysis at Scale: Leveraging Tracking Data in Sports