Run a benchmark they said. It will be fun they said.
summary
A practical recounting of building benchmarks for the skrub library by using publicly available datasets from old Kaggle competitions to choose default parameters. Covers the unexpected scale and complications encountered while running those benchmarks and the lessons learned from that process.
AI-written summary · about
related talks
44mKeynote: The neglected art of a relevant benchmark and why you need one
20mSkrub DataOps, Probabl
31mMove beyond academia: Introducing an industry-first tabular benchmark
16mBenchmarks Are Memes: How What We Measure Shapes AI—and Us
11mSotabench for Reproducible Research
21mBen Hamner, Co-founder and CTO, Kaggle
40m[Full Workshop] Building Metrics that actually work
23mBenchmarking Machine Learning Workloads on Kubeflow