google-research / google-research/tabfm

Negative softmax_temperature silently inverts predictions; zero yields all-nan probabilities

Open Beginner friendly
#93 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.6k
Forks
270
Avg merge
1d 7h
Merged PRs (30d)
1

Description

`softmax_temperature` is used as a divisor with no guard (`tabfm/src/classifier_and_regressor.py:3372`):

```python
x = x / temperature
```

and the estimator does not define sklearn's `_parameter_constraints`, so nothing validates it on the way in either. I checked — there is no `_parameter_constraints` anywhere in the tree, which means scikit-learn's own parameter validation never runs for any of the estimator's parameters.

Reproduced on `b15593e4c1111ddb5f4f30dd2957df2edbaa04ca` against `TabFMClassifier.softmax` directly, so no checkpoint is needed:

```
temperature=0.9 -> [[0.6895, 0.227, 0.0835]] sum=1.0000 [no warning]
temperature=0.0 -> [[nan, nan, nan]] sum=nan [RuntimeWarning]
temperature=-1.0 -> [[0.0961, 0.2613, 0.6426]] sum=1.0000 [no warning]
temperature=1e-300 -> [[1.0, 0.0, 0.0]] sum=1.0000 [no warning]
```

The `-1.0` row is the one I would flag. The probabilities still sum to 1.0 and nothing warns, but the ordering is reversed — the input logits rank class 0 highest, and the output ranks class 2 highest. A negative temperature silently inverts the prediction rather than failing.

`0.0` is milder in that it at least emits a `RuntimeWarning`, but the result is all-`nan` probabilities, and a warning is easy to miss inside a fit loop.

A one-line check that `softmax_temperature > 0` would cover both. If you would rather adopt sklearn's `_parameter_constraints` for the estimator as a whole, that would cover this and the rest of the constructor arguments in one go — happy to send either.

Disclosure: I used an AI assistant to help find this. I ran the reproduction myself.

Contributor guide

Open the contributing guide

Research direction

Start at tabfm/src/classifier_and_regressor.py:3372 and inspect TabFMClassifier.softmax and the estimator initialization. Reproduce the temperature cases from the issue, then add regression coverage for invalid temperatures and confirm valid temperatures remain normalized without warnings. Done means non-positive temperatures are rejected before division.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.