Trusting AI in high integrity systems
summary
Defines which specific functions need fail-safe behaviour and the exact contexts (motorway vs rural, mph vs kph), then applies standard hazard brainstorming, likelihood×severity scoring and mitigation to reach SIL 4 or lower levels; flags tool confidence for development and data used to train AI; cites historical failures to show scale: Clapham 35 dead, Therac-25 six killed, and a scheduling algorithm linked to Paddington 31 dead 417 injured as examples of modifications or automation that become catastrophic when context, tooling or human interaction are ignored.
AI-written summary · about
chapters (9)
- 0:00High integrity systems intro
- 1:33High integrity functions
- 3:17Software failure lessons
- 6:04Risk assessment processes
- 9:10Safety integrity levels
- 11:21Training AI trustworthiness
- 12:46Human-system safety risks
- 14:56Trusting AI principles
- 16:01Summary & Q&A
related talks
33mHuman Factors in the Age of AI Ops: Re-Engineering Trust between Humans and...
32mAI for the Masses Building AI Products that Users Trust
47mResponsible AI: A Hands-On Approach
45mWhat Security Measures Are Necessary to Trust in AI?
16mTrust by Design: Building Reliable AI for Sensitive Environments
52mThe Risky Business, of AI Illiteracy
41mThe High Stakes Security of AI
37mAI Under Control: accelerating safely