SAVE: Software-Implemented Fault Tolerance for Model Inference against GPU Memory..
summary
Claims software-only SAVE keeps model accuracy even under 4,000 GPU memory bit flips while adding less than 9% runtime overhead and under 1% correctness impact. It identifies vulnerable bits with static range and bit‑attribution analysis, places high‑risk tensor blocks into tiny reliable SRAM, and offloads checks to the CPU with asynchronous CPU verification plus a lightweight in‑kernel range check and a mixed‑precision CPU recompute for recovery. Tested against real bit‑flip traces and on RTX 1490, it restarts from the affected layer to recover.
AI-written summary · about
chapters (8)
- 0:00SAVE introduction
- 2:09Existing solutions for bit flips
- 3:53SAVE’s key insights
- 6:00SAVE workflow stages
- 9:15Allocation and verification
- 10:42Error handling and latency
- 12:09Performance overhead results
- 13:59Conclusion and Q&A
related talks
23mStability in Large Model Training: Practices in Software and Hardware Fault Self-Healing
23mSAGE: Software-based Attestation for GPU Execution
17mFaith: An Efficient Framework for Transformer Verification on GPUs
13mByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
28mDark Silicon: Unmasking GPU Threats in the Age of AI
44mGPU-less, Trust-less, Limit-less: Reimagining the Confidential AI Cloud
21mEnhancing Runtime Reliability in LLM Training via Fine-Grained Observability
14mHoneycomb: Secure and Efficient GPU Executions via Static Validation