~/talk

Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod

links:none on file
slides:none on file

BenchPod is a vLLM-based benchmarking dashboard that lets teams pick configs from data instead of guessing, showing a live demo of BenchPod, running Kimi K3 on a B300 and comparing concurrency up to 192. Changing max batch tokens cuts inter-token latency significantly while time-to-first-token barely changes. The system reports ~9 minute cold starts for K3, has self-healing multi-node runs, and produces measurable wins that can be promoted to production.

AI-written summary · about

  1. 0:00Intro
  2. 0:29What Runpod does
  3. 1:32Everyone's guessing at vLLM configs
  4. 2:53Version and CUDA gotchas
  5. 3:48BenchPod: benchmarking on top of vLLM
  6. 6:07Running Kimi K3 on a B300
  7. 7:40Demo
  8. 10:01Tuning max batch tokens
  9. 13:34From R&D to production
  10. 14:21Do you need the frontier model?
~/search
type to search 6,518 conferences, 143,918 speakers and 192,133 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
← →
on a talk: 5 seconds back / forward
0–9
on a talk: jump to 0%–90% of the recording
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.