Lightning-AI / Lightning-AI/torchmetrics
Functional nominal metrics raise RuntimeError when category labels do not start at 0
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 526
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 5
Description
## 🐛 Bug
Functional nominal metrics (`cramers_v`, `pearsons_contingency_coefficient`, `theils_u`, `tschuprows_t`) raise a `RuntimeError` during confusion matrix calculation when categorical inputs contain non-zero-based or non-contiguous category labels (e.g., labels `[1, 2]` or `[5, 6]`).
### To Reproduce
Code sample
```python
import torch
from torchmetrics.functional.nominal import cramers_v
preds = torch.tensor([5, 6, 5, 6, 5, 6, 5, 6])
target = torch.tensor([5, 6, 5, 5, 6, 6, 5, 6])
# Raises RuntimeError: shape '[2, 2]' is invalid for input of size 19
cramers_v(preds, target)
```
Environment
- TorchMetrics version: 1.9.0 (latest master)
- Python & PyTorch Version: Python 3.10+, PyTorch 2.x
- Any other relevant information such as OS: All platforms (Linux, macOS, Windows)
### Additional context
The functional API infers `num_classes = len(torch.cat([preds, target]).unique())` but passes the unmapped raw values to `_multiclass_confusion_matrix_update`, which expects values strictly in `[0, num_classes - 1]`. Any values outside this range cause invalid bin indexing and reshape failures. Additionally, `_compute_chi_squared` performs in-place float addition on integer `confmat` tensors, causing type cast issues. Remapping categorical labels to contiguous 0-based indices before updating the confusion matrix resolves this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the functional nominal metric entry points for cramers_v, pearsons_contingency_coefficient, theils_u, and tschuprows_t, then trace their use of _multiclass_confusion_matrix_update and _compute_chi_squared. Reproduce the supplied labels [5, 6] and verify that all four metrics calculate successfully without reshape or tensor type-cast errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100