google-research / google-research/tabfm
Negative softmax_temperature silently inverts predictions; zero yields all-nan probabilities
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 270
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 1
Description
`softmax_temperature` is used as a divisor with no guard (`tabfm/src/classifier_and_regressor.py:3372`):
```python
x = x / temperature
```
and the estimator does not define sklearn's `_parameter_constraints`, so nothing validates it on the way in either. I checked — there is no `_parameter_constraints` anywhere in the tree, which means scikit-learn's own parameter validation never runs for any of the estimator's parameters.
Reproduced on `b15593e4c1111ddb5f4f30dd2957df2edbaa04ca` against `TabFMClassifier.softmax` directly, so no checkpoint is needed:
```
temperature=0.9 -> [[0.6895, 0.227, 0.0835]] sum=1.0000 [no warning]
temperature=0.0 -> [[nan, nan, nan]] sum=nan [RuntimeWarning]
temperature=-1.0 -> [[0.0961, 0.2613, 0.6426]] sum=1.0000 [no warning]
temperature=1e-300 -> [[1.0, 0.0, 0.0]] sum=1.0000 [no warning]
```
The `-1.0` row is the one I would flag. The probabilities still sum to 1.0 and nothing warns, but the ordering is reversed — the input logits rank class 0 highest, and the output ranks class 2 highest. A negative temperature silently inverts the prediction rather than failing.
`0.0` is milder in that it at least emits a `RuntimeWarning`, but the result is all-`nan` probabilities, and a warning is easy to miss inside a fit loop.
A one-line check that `softmax_temperature > 0` would cover both. If you would rather adopt sklearn's `_parameter_constraints` for the estimator as a whole, that would cover this and the rest of the constructor arguments in one go — happy to send either.
Disclosure: I used an AI assistant to help find this. I ran the reproduction myself.
Contributor guide
Research direction
Start at tabfm/src/classifier_and_regressor.py:3372 and inspect TabFMClassifier.softmax and the estimator initialization. Reproduce the temperature cases from the issue, then add regression coverage for invalid temperatures and confirm valid temperatures remain normalized without warnings. Done means non-positive temperatures are rejected before division.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100