SOTER: Guarding Black-box Inference for General Neural Networks at the Edge
summary
SOTER is a system implemented in Python with Graphene-SGX that uses parameter-morphed operators to run most linear-heavy layers on untrusted Nvidia GPUs while keeping scalars and final restoration inside SGX. It evaluates six popular models (CV and NLP), achieves 1×–4× lower latency versus full-TEE shielding and matches partition baselines, retains original accuracy (no retraining), and detects tampering with an oblivious fingerprinting scheme that yields high-probability integrity checks at runtime.
AI-written summary · about
chapters (6)
- 0:00Start
- 0:23Secure and High Performance Influence System for General Neural Networks
- 1:25Basic Workflow
- 5:03Te Shielding Approach
- 8:22Recap of Dealing Mode Architecture
- 15:23End-to-End Performance
related talks
28mGenerative AI + Proprietary Data
19mThe Small Model Infrastructure Nobody Built (So We Did)
27mAdvancements in optimizing ML Inference at CERN
37mDeep Learning Neural Network Acceleration at the Edge
44mGPU-less, Trust-less, Limit-less: Reimagining the Confidential AI Cloud
23mSAGE: Software-based Attestation for GPU Execution
18mTelekine: Secure Computing with Cloud GPUs
21mXorbits Inference: Model Serving Made Easy