The Foundation Model Revolution for Tabular Data
summary
Claims state-of-the-art on small tabular tasks, beating XGBoost in 96% of use cases and reaching similar accuracy with 50% of the training data. Model ingests up to 10,000 rows and 500 features, handles categorical values, missing and uninformative features, and predicts via a single forward pass. Architecture uses axial attention over rows and columns, training uses hundreds of millions of synthetic datasets from structural causal priors. Limits: needs a GPU, slower than trees for instant inference, and struggles with millions of rows.
AI-written summary · about
chapters (9)
- 0:00Foundation models for tabular data
- 5:56TPFN dataset inputs
- 9:47AutoML to forward pass
- 13:36TPFN vs classical ML
- 16:50Missing values & robustness
- 18:14TPFN for time series
- 20:21Fine-tuning foundation models
- 23:11TPFN limitations
- 24:59Q&A
related talks
01h 28mHands-On with Tabular Foundation Models
30mTabular data Transformed? TabPFN Brings Deep Learning to the Table
05mFoundationDB as Datastore for Training Deep Learning Models
01h 03mTowards Foundation Models for Graphs
33mFrontiers of Foundation Models for Time Series
41mBetter Than Deep Learning: Gradient Boosting Machines (GBM)
29mData management for enterprise-grade foundation models.
25mHumans helping machines help humans run machines