ML Automoderation System for LLM-Augmented Online Platforms
summary
Production pipeline moves raw chats through cleaning, deduplication, sampling, hash journal checks and few‑shot retrieval, then uses LLM pre-annotation as a fast intern that outputs labels, explanations and highlights for human validation; humans curate a golden set for training lightweight BERT‑style online models, LLM reasoning layers and reviewer queues. Thresholds are category‑specific and operational, reviewers are prioritized, and monitoring tracks drift. Reported scale: millions of messages per day with 60% review reduction and 90%+ violations caught.
AI-written summary · about
chapters (16)
- 0:00Intro and Goal
- 2:07Why Moderation Changed
- 3:47Dating Safety Stakes
- 5:20Imbalance and Risk
- 6:53Policy as Specification
- 8:32System Pipeline Overview
- 10:12Data Alignment Layer
- 11:39Deduplication Pitfalls
- 13:06LLM Prelabeling Loop
- 15:48Gray Zones and Context
- 17:23Hybrid Moderation Stack
- 20:05Decisioning and Thresholds
- 22:52Evaluation and Monitoring
- 25:28Drift and Adversaries
- 26:54Impact Metrics Caution
- 28:23Key Takeaways and Outro
related talks
26mModern NLP for Proactive Harmful Content Moderation
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
29mLocal LLM fine-tuning: a practical example
30mLLM-Based Recommendation Systems
19mLLM-Enhanced Multimodal AI: Revolutionizing Audio/Video Interaction
15mML Algorithms for Trust and Safety
34mScaling Trust
59mWorkshop: Choosing the Best Open Source LLM for Your Application