Building Once, Running Everywhere: The Future of Portable AI Applications
summary
Claims that one binary can run identically everywhere are false; instead the stack uses portable AI runtimes plus hardware abstraction, intelligent compilation and automated testing to adapt models. Demonstrations show 600 tokens/sec dropping to 15, a 300 ms GPS lag breaking a smartwatch UX, and teams tracking ~164 parameter permutations. Testing deploys builds across representative targets (cloud, phone, NPU, embedded), measuring accuracy, latency, memory, power and thermal to keep results inside a multi-dimensional quality envelope defined by five axes.
AI-written summary · about
chapters (34)
- 0:00Building Once, Running Everywhere: Portable AI Applications
- 1:31Speaker Intro - Prashant Sharma, Head of Developer Relations at Arm
- 2:32Why Write Once, Run Everywhere Got Harder, Not Easier
- 4:08"It Works on My Machine" - The Phrase We Never Killed
- 4:40Abstraction Layers, VMs and the Web - Each Got Us Closer
- 5:12AI Is Escaping the Data Centre Into Phones, Cars, Cameras, Robots
- 5:43Can I Build an Intelligent Application Once and Run It Everywhere?
- 6:16The Tester's Harder Question: If It Runs Everywhere, Does It Still Work?
- 6:48The Exploding AI Compatibility Matrix
- 7:49Model x Framework x Runtime x OS x CPU/GPU/NPU x Precision x Device
- 8:21Portability Used to Be Binary - AI Makes It Continuous
- 8:51It Runs, It Doesn't Crash, and 600 Tokens a Second Becomes 15
- 9:21The Smartwatch GPS Example: 300 Milliseconds That Break the Experience
- 9:51Same Model, Different Machines, Different Answers
- 10:52Portability Does Not Mean Identical - It Means Acceptable
- 11:54Can We Make AI Deliver an Acceptable Experience Everywhere?
- 12:26Hardware Should Become Boring: The Emerging Abstraction Stack
- 14:00The More Complexity You Hide, the More Responsibility Moves to Validation
- 14:32What "Correct" Even Means Now: 94% to 93.8% to 91%
- 15:34The Multi-Dimensional Quality Envelope
- 16:05Five Axes: Correctness, Performance, Efficiency, Quality, Portability
- 16:35Every Optimisation Becomes an Experiment
- 17:06You Cannot Manually Test 164 Versions of Parameters
- 17:36Testing Deployed Across Representative Hardware Targets
- 18:06Helping Developers Navigate Trade-Offs, Not Just Detect Failures
- 19:07AI Forces Hardware Back Into the Testing Conversation
- 19:37A Five-Minute Test Tells You Nothing About Thermal Behaviour
- 20:08Developer Intent as the Contract the Platform Verifies
- 20:40What Build Once, Run Everywhere Really Means
- 21:40The Automated Testing That Keeps You Inside the Quality Envelope
- 22:10Closing: Build Once, Test Everywhere, Run Anywhere With Confidence
- 24:15Q&A: Does Human in the Loop Still Work at Enterprise Scale?
- 25:47Q&A: Building a Context and Capability Aware Agent
- 27:51Q&A: What Changes in Software Engineering Over the Next 5-10 Years
related talks
21mBeyond the Chatbox: What Changes When AI Runs Where You Are
01h 07m📚 Building Portable Large Models on Heterogeneous AI Accelerators Wit...
01h 19mWORKSHOP 2: AI FOR EVERYONE
39mRunning On-Device AI With Qualcomm AI Engine Using LiteRT & ExecuTorch
06mThe Building Blocks of Agentic Al
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution
30mCloud? No Thanks! I’m Gonna Run GenAI on My AI PC
59mKeynote: AI without the BS, for humans