SELECT instance FROM cloud WHERE workload = ? ORDER BY cost
summary
Spare Cores Navigator is a Python‑queryable benchmark dataset covering 5,000+ instance types and >500 workloads with standardized performance and cost‑efficiency metrics, built by automatically provisioning instances via public GitHub Actions and running benchmarks. LLM inference uses Llama CPP (tested on Llama 3.3 70B and gamma 2B) and produces surprising results like CPU‑only machines in the top‑25 by price‑performance. The project publishes SQLite and APIs, spent about 10,000 euros on LLM runs, and admits gaps from quotas, spot pricing, and missing measurements.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:11Cloud server database
- 5:47Hidden costs in pricing
- 9:00Benchmarking challenges
- 10:23Speaker's background and project motivation
- 12:47Spare Course features
- 13:28Data collection process
- 14:32Live demo: server selection
- 17:45Server specs and benchmarks
- 21:36Comparing cloud servers
- 22:39API and Python access
- 24:07Licensing details
- 24:49LLM benchmarking
- 30:25Best server for LLM
- 31:32Benchmarking tech and disclaimers
- 33:17Benchmark cost and feasibility
- 34:00Q&A
related talks
17mOptimizing Cloud Compute Resources with Spare Cores
35mLocal LLMs on low end hardware - a practical perspective
36mMigrating Applications to the Cloud in a Cost Effective Manner
33mLearn How to Run an LLM Inference Performance Benchmark on NVIDIA GPUs
19mServerless inferencing: an infrastructure point of view
50mKa-Ching! How Enterprises Measure LLM Performance and Cost
20mSecure, private LLMs in your cloud with Anyscale Endpoints
29mMulti-Accelerator PyTorch Serving With NxD Inference and vLLM