"How AI improved our performance by 20%" and other bedtime stories
summary
A 25‑year‑old safety‑critical software firm runs two year‑long R&D tracks from May 2023, conducts 25 interviews, tests 30+ tools and identifies 80 relevant use cases, then launches an “AI Shift” with ~10 initiatives. Developers in a controlled pilot (79 people) report a 27% perceived productivity uplift; separate studies warn high‑performers may gain ~20% while low‑performers can fall ~10%. Solutions include Teams‑integrated RAG retrieval, vector DBs, traceable sources and strict privacy controls.
AI-written summary · about
chapters (23)
- 0:00Introduction and setup
- 1:03AI hype and reality
- 2:07About Critical Software
- 3:10Generative AI explosion
- 4:33AI performance claims debunked
- 5:36Critical Software’s AI research goals
- 6:59AI Assistant project findings
- 8:04Project X and software lifecycle
- 9:30AI Shift initiative launch
- 10:56Three-stage AI Shift plan
- 12:01Use cases and concerns
- 14:11Evidence-based adoption framework
- 15:56Balancing speed and deep tech
- 16:58Knowledge retrieval use cases
- 18:44Critical Titans AI assistant demo
- 21:16Demo walkthrough
- 26:32Beyond generative AI
- 27:57AI Shift communication strategy
- 29:01Performance claims corrected
- 30:06Developer productivity results
- 31:50AI Letter of Intent principles
- 34:19AI Shift end goals
- 37:17Q&A session
related talks
18mHumans + AI: The Future of Software Development Teams
27mThe state of AI in software development: Insights across 400+ organizations
44mSpeed Is Easy, Trust Is Hard: AI's Real Impact on Developer Productivity
17mLeadership in AI-assisted engineering
32mAI and Engineering Closing the Gap Between Expectations and Reality 1
12mHow is AI reshaping tech teams
18mAI Unleashed: Forging the Future of Rapid Innovation and Growth
18mDoes AI Actually Boost Developer Productivity? (100k Devs Study)