Faux Data, Real Defense: ML advancements in data synthesis
summary
Generates adversarial Kubernetes API server audit logs with LLMs using one-shot prompting, evaluates four models by fidelity, reproducibility, and accuracy, and validates synthetic records against existing detection rules. Uses LangChain output parser to force YAML-like schema. Most models produce ~10–15% nulls (Llama 70b near single digits), Mixl gives the best reproducibility, GPT-3.5 variants detect node-proxy escalation well, while privileged exec and pod-token theft remain hard; synthetic data helps but does not replace red teams.
AI-written summary · about
chapters (14)
- 0:00Introduction and speaker welcome
- 1:06Threat detection challenges and synthetic data
- 3:37Kubernetes audit logs for testing
- 5:04Experimental setup and attack techniques
- 6:31Evaluating synthetic data quality
- 7:58Prompt design for data generation
- 9:00Fidelity results of models
- 10:53Fidelity beyond null values
- 12:16Reproducibility of synthetic data
- 14:44Accuracy for detection rules
- 17:14Model performance comparison
- 18:40Limitations and future work
- 20:04Next steps and model expansion
- 22:11Q&A session
related talks
30m🤯 No data? No problem! Synthetic data to the rescue
23mSecurity and auditing tools in Large Language Models (LLM)
25mAttack, Defense & Danger in the Age of AI
36mArchitecting Data: A Deep Dive Into the world of Synthetic Data
01h 04mCan you trust your (large language) model?
28mFlawed ML Security: Mitigating Security Vulnerabilities in Data & Machine...
21mEnd-to-end secure ML Development
30mOwned by Statistics: Using Kubeflow to Defend vs Attacks on Your ML Models