Detoxification of LLMs using TrustyAI Detoxify and HuggingFace SFTTrainer
summary
Claims a no-annotator detox pipeline: TrustyAI Detoxify rephrase via token disagreement by scoring next-token probability differences between two T‑models, masking top‑disagreement tokens and replacing them with non‑toxic alternatives; then applies supervised finetuning using HuggingFace SFTTrainer with PEFT LoRA + 4‑bit quantization to avoid catastrophic forgetting and cut storage/compute. Evaluation shows the detoxified model has lower mean and standard deviation toxicity scores than a conventionally fine‑tuned model and produces non‑toxic generations for sensitive prompts; code and examples are linked in the repo.
AI-written summary · about
chapters (7)
related talks
26mModern NLP for Proactive Harmful Content Moderation
34mTerrible tokenizer troubles in large language models
29mLocal LLM fine-tuning: a practical example
29mLies, damned lies and large language models
25mFinetuning Large Language Models (LLMs) for Security Log Detections
34mScaling Trust
59mRed Teaming Large Language Models
41mTruth Checker: Generative Large Language Models and Hallucinations