Jay Marshall, Neural Magic | AWS Startup Showcase S3E1
summary
Neural Magic shows software that decouples model execution from expensive accelerators: SparseML optimizes and sparsifies models, SparseZoo publishes pre-optimized models, and the DeepSparse runtime runs them on x86 CPUs and now AWS Graviton ARM (early access). Claim: OPT-style LLMs can be sparsified >50% to run on a single GPU; a retail customer doubles camera count on the same compute. Tooling is open source and DeepSparse is in the AWS Marketplace.
AI-written summary · about
