What I Learned Building an AI Documentation Auditor (And What It Found in 500+ Pages of Docs)
summary
Built an AI auditor that continuously scans docs and it actually ran over 500+ pages across 100+ portals and finds concrete failures: deprecated code examples, broken API endpoints, and version mismatches that make a manual three‑month audit obsolete within weeks. It reports five concrete quality dimensions—relevance, freshness, clarity, accuracy, completeness—and shows AI reliably flags syntax errors but hallucinates about clarity. Practical fixes include adding LLMs.txt, exposing content as JSON-LD and markdown, pre-rendering text, and adding CI code validation.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:50AI readiness for docs
- 3:38AI documentation auditor
- 7:33AI-ready docs failures
- 12:29Content quality for AI
- 16:01Continuous docs audits
- 17:26Freshness in AI era
- 22:00Prioritizing fixes
- 25:57High-impact optimizations
- 30:08Key takeaways
- 33:42Q&A
related talks
24mDocumentation and AI: How to write right now
15mAI's Impact on Development, Post-Production & Support Journeys
14mAI for Web3 Security: Real-World Use Cases — Dr. Stefan Beyer
39mDocs AI Tooling is Better (and Better for Us) than You Think
27mAI-First Software Maintenance & AI-First Software Delivery
41mSurviving and Thriving with AI in QA
39mBack to Basics: Cutting Through the AI-Generated Code Noise
43mHow to Use AI to Build Accessible Products