google-research / google-research/tabfm

Multi-Quantile / Distributional Regression Support

Open
#31 7 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.6k
Forks
270
Avg merge
1d 7h
Merged PRs (30d)
1

Description

**Motivation**
Currently, TabFMRegressor focuses on point-estimate regression (minimizing MSE/RMSE). However, point estimates discard predictive uncertainty, which is critical for real-world tabular data plagued by heavy tails or heteroskedasticity (see Image 1 for an example of suboptimal modeling with point estimates).

We recently introduced ScoringBench ([scoringbench.com](https://scoringbench.com/)), a framework designed to evaluate tabular models under proper scoring rules like CRPS and interval scores. The theoretical motivation and empirical shifts in model rankings are detailed in our accompanying preprints:

[ScoringBench: A Benchmark for Evaluating Tabular Foundation Models with Proper Scoring Rules (arXiv:2603.29928)](https://arxiv.org/abs/2603.29928)

[Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules (arXiv:2603.08206)](https://arxiv.org/abs/2603.08206)

To rigorously evaluate TabFM's zero-shot uncertainty probabilistic predictive performance on this benchmark, the model needs the ability to output a predictive distribution rather than a single point estimate

**Proposed Feature: Multi-Quantile Regression**
We propose extending TabFM to support non-parametric distributional regression e.g. via multi-quantile estimation (as in tabiclv2):

API Extension: Allow users to request an array of target quantiles, e.g., predict_quantiles(X_test, quantiles=[0.01, ... 0.05, 0.25, ... 0.5, 0.75, 0.95]) (in a fine grid).

Loss Optimization: Adapt the output head or decoding strategy to optimize pinball loss across the specified spectrum simultaneously.

Since the integral of the pinball loss approximates the Continuous Ranked Probability Score (CRPS), this directly optimizes the model for proper scoring rules.

**Evaluation**
Adding this feature would allow us to benchmark TabFM on the ScoringBench suite, providing a direct, statistically rigorous comparison of TabFM's probabilistic performance against baselines like TabPFN and quantile XGBoost and many other conditional density estimators.

Contributor guide

Open the contributing guide

Research direction

Start by locating TabFMRegressor and its prediction/output path, then compare the proposed predict_quantiles API with the tabiclv2 multi-quantile approach. Work out how requested quantiles and pinball-loss optimization fit the output head or decoding strategy. Done means evaluating probabilistic predictions on ScoringBench with CRPS and interval scores against the listed baselines.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.