Synthetic data that meets enterprise compliance needs
summary
An agentic LLM pipeline generates domain-rich, visually realistic enterprise documents using tool calls (charts/tables via image tools, open-source faker library for contextual PII), an LLM-as-judge for coherence, and ML Commons safety checks for 14 toxicity categories. It produces 50,000+ enterprise-like documents across finance, healthcare and legal, yields significant F1 gains for NER and classification, and handles long PDFs (up to ~100k tokens); remaining risks are indistinguishability, mode collapse, and LLM-sourced hallucinated real names.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:03Enterprise compliance challenges
- 4:11Synthetic data with agents and LLMs
- 6:19Synthetic data pipeline overview
- 8:02Planning document generation
- 9:28Document construction and PII insertion
- 10:29Quality and safety checks
- 10:51Generated enterprise documents
- 15:27Evaluating synthetic data quality
- 17:34Benchmarking synthetic data
- 18:18Key learnings and challenges
- 20:01Q&A
related talks
33mDeep research at scale
28mAnonymization in the LLM Era: No Room for Compromise
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
52mAnatomy of an LLM
29mAutomating Privacy Reviews with LLMs: Challenges & Opportunities
10mWhen Privacy Guarantees Meet Pre-Trained LLMs: A Case Study in Synthetic Data
30mBuild an AI knowledge fabric for your organisation
50mDehumanizing Agents: Why Explainability is Crucial in the LLM Era