Luis Ceze & Anna Connolly, OctoML | AWS Startup Showcase S3 E1
summary
OctoML claims automated model-to-production optimization that cuts cloud serving cost and avoids hardware lock-in, offering a one-line PyTorch native profiler and an Octopod beta. A customer benchmark shows they saved 25% on T4 and by switching to A10 plus optimizations achieved sub two milliseconds latency, unlocking new UX. An internal GPT-J scenario estimates a conservative 3000 words/user/day at ~$0.05 per session scales to about $6 million a year for 1M daily users.
AI-written summary · about
