google-research / google-research/tabfm
Multi-Quantile / Distributional Regression Support
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 270
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 1
Description
**Motivation**
Currently, TabFMRegressor focuses on point-estimate regression (minimizing MSE/RMSE). However, point estimates discard predictive uncertainty, which is critical for real-world tabular data plagued by heavy tails or heteroskedasticity (see Image 1 for an example of suboptimal modeling with point estimates).
We recently introduced ScoringBench ([scoringbench.com](https://scoringbench.com/)), a framework designed to evaluate tabular models under proper scoring rules like CRPS and interval scores. The theoretical motivation and empirical shifts in model rankings are detailed in our accompanying preprints:
[ScoringBench: A Benchmark for Evaluating Tabular Foundation Models with Proper Scoring Rules (arXiv:2603.29928)](https://arxiv.org/abs/2603.29928)
[Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules (arXiv:2603.08206)](https://arxiv.org/abs/2603.08206)
To rigorously evaluate TabFM's zero-shot uncertainty probabilistic predictive performance on this benchmark, the model needs the ability to output a predictive distribution rather than a single point estimate
**Proposed Feature: Multi-Quantile Regression**
We propose extending TabFM to support non-parametric distributional regression e.g. via multi-quantile estimation (as in tabiclv2):
API Extension: Allow users to request an array of target quantiles, e.g., predict_quantiles(X_test, quantiles=[0.01, ... 0.05, 0.25, ... 0.5, 0.75, 0.95]) (in a fine grid).
Loss Optimization: Adapt the output head or decoding strategy to optimize pinball loss across the specified spectrum simultaneously.
Since the integral of the pinball loss approximates the Continuous Ranked Probability Score (CRPS), this directly optimizes the model for proper scoring rules.
**Evaluation**
Adding this feature would allow us to benchmark TabFM on the ScoringBench suite, providing a direct, statistically rigorous comparison of TabFM's probabilistic performance against baselines like TabPFN and quantile XGBoost and many other conditional density estimators.
Contributor guide
Research direction
Start by locating TabFMRegressor and its prediction/output path, then compare the proposed predict_quantiles API with the tabiclv2 multi-quantile approach. Work out how requested quantiles and pinball-loss optimization fit the output head or decoding strategy. Done means evaluating probabilistic predictions on ScoringBench with CRPS and interval scores against the listed baselines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100