Lightning-AI / Lightning-AI/torchmetrics
davies_bouldin_score returns a wrong value for off-origin float64 data (float32 buffers)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 526
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 5
Description
## 🐛 Bug
`davies_bouldin_score` and `calinski_harabasz_score` allocate their accumulators without a `dtype`, so the buffers are `float32` regardless of the input. `dunn_index` — same family, same input signature — preserves the input dtype, so the clustering metrics disagree with each other.
For `davies_bouldin_score` this is not only a dtype-label problem. The `float32` centroid buffer causes catastrophic cancellation for data that is not centred near zero, and the returned score becomes meaningless.
### To Reproduce
```python
import torch
from torchmetrics.functional.clustering import calinski_harabasz_score, davies_bouldin_score, dunn_index
data = torch.rand(300, 3, dtype=torch.float64)
labels = torch.arange(300) % 4
print(calinski_harabasz_score(data, labels).dtype) # torch.float32 <--
print(davies_bouldin_score(data, labels).dtype) # torch.float32 <--
print(dunn_index(data, labels).dtype) # torch.float64
```
### The serious part: `davies_bouldin_score` on off-origin data
Same `float64` data, shifted away from the origin, compared against the identical algorithm run with `float64` buffers:
```python
data = torch.rand(300, 3, dtype=torch.float64) + offset
labels = torch.arange(300) % 4
```
| `offset` | `davies_bouldin_score` | `float64` reference | relative error |
|---|---|---|---|
| `0` | 27.0118026733 | 27.0118054968 | 1.0e-07 |
| `1e3` | 27.0143089294 | 27.0118054967 | 9.3e-05 |
| `1e5` | 29.3704795837 | 27.0118054947 | **8.7e-02** |
| `1e7` | **1.7268073559** | 27.0118055903 | **9.4e-01** |
At `offset=1e7` the metric returns `1.73` where the correct value is `27.01`. The clustering structure is identical in every row — only the origin moved.
Off-origin coordinates are ordinary: geographic positions, epoch timestamps, unnormalised sensor readings. A user who reaches for `float64` precisely because their data has a large dynamic range currently gets it silently discarded.
### Cause
Buffers are allocated without `dtype`, so they default to `float32`:
`functional/clustering/davies_bouldin_score.py`:
```python
intra_dists = torch.zeros(num_labels, device=data.device) # float32
centroids = torch.zeros((num_labels, dim), device=data.device) # float32
...
centroids[k] = cluster_k.mean(dim=0) # float64 mean, narrowed on assignment
intra_dists[k] = (cluster_k - centroids[k]).pow(2.0).sum(dim=1).sqrt().mean()
```
Writing a `float64` centroid into the `float32` buffer rounds it to about 7 significant digits. At `offset=1e7` the stored centroid is wrong by roughly `1e0`, while the true intra-cluster spread is roughly `0.3` — so `cluster_k - centroids[k]` is dominated by the rounding error rather than by the data.
`functional/clustering/calinski_harabasz_score.py`:
```python
between_cluster_dispersion = torch.tensor(0.0, device=data.device) # float32
within_cluster_dispersion = torch.tensor(0.0, device=data.device) # float32
```
Here only the running accumulation is narrowed — the per-cluster terms are still computed in the input dtype — so the damage is bounded at about `1e-7` relative rather than being catastrophic. It is still a silent `float32` ceiling on a `float64` input.
Both also return `torch.tensor(..., dtype=torch.float32)` explicitly from their degenerate branches, which would stay inconsistent even after the buffers are fixed.
### Expected behavior
The accumulators should be allocated in the input dtype, so `float64` in gives `float64` out and the computation is actually carried out at the requested precision — matching `dunn_index`.
### Environment
- TorchMetrics: `master` (commit 37a80809)
- PyTorch: 2.13.0+cpu, Python 3.13, Windows
### Related
Same class of problem as #3465 (`spearman_corrcoef` / `kendall_rank_corrcoef`) but a different mechanism — that one is integer division falling back to the default dtype, this one is untyped buffer allocation. Reporting separately since the fix and the blast radius are different.
I have a fix and regression tests ready and am happy to open a PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read functional/clustering/davies_bouldin_score.py and functional/clustering/calinski_harabasz_score.py, then compare their dtype handling with dunn_index. Run the existing clustering tests and add regression coverage for float64 inputs, including off-origin data. Done means the affected metrics preserve the input dtype and return the expected values in degenerate cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100