Lightning-AI / Lightning-AI/torchmetrics

davies_bouldin_score returns a wrong value for off-origin float64 data (float32 buffers)

Open
#3,467 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.5k
Forks
526
Avg merge
6d 11h
Merged PRs (30d)
5

Description

## 🐛 Bug

`davies_bouldin_score` and `calinski_harabasz_score` allocate their accumulators without a `dtype`, so the buffers are `float32` regardless of the input. `dunn_index` — same family, same input signature — preserves the input dtype, so the clustering metrics disagree with each other.

For `davies_bouldin_score` this is not only a dtype-label problem. The `float32` centroid buffer causes catastrophic cancellation for data that is not centred near zero, and the returned score becomes meaningless.

### To Reproduce

```python
import torch
from torchmetrics.functional.clustering import calinski_harabasz_score, davies_bouldin_score, dunn_index

data = torch.rand(300, 3, dtype=torch.float64)
labels = torch.arange(300) % 4

print(calinski_harabasz_score(data, labels).dtype) # torch.float32 <--
print(davies_bouldin_score(data, labels).dtype) # torch.float32 <--
print(dunn_index(data, labels).dtype) # torch.float64
```

### The serious part: `davies_bouldin_score` on off-origin data

Same `float64` data, shifted away from the origin, compared against the identical algorithm run with `float64` buffers:

```python
data = torch.rand(300, 3, dtype=torch.float64) + offset
labels = torch.arange(300) % 4
```

| `offset` | `davies_bouldin_score` | `float64` reference | relative error |
|---|---|---|---|
| `0` | 27.0118026733 | 27.0118054968 | 1.0e-07 |
| `1e3` | 27.0143089294 | 27.0118054967 | 9.3e-05 |
| `1e5` | 29.3704795837 | 27.0118054947 | **8.7e-02** |
| `1e7` | **1.7268073559** | 27.0118055903 | **9.4e-01** |

At `offset=1e7` the metric returns `1.73` where the correct value is `27.01`. The clustering structure is identical in every row — only the origin moved.

Off-origin coordinates are ordinary: geographic positions, epoch timestamps, unnormalised sensor readings. A user who reaches for `float64` precisely because their data has a large dynamic range currently gets it silently discarded.

### Cause

Buffers are allocated without `dtype`, so they default to `float32`:

`functional/clustering/davies_bouldin_score.py`:

```python
intra_dists = torch.zeros(num_labels, device=data.device) # float32
centroids = torch.zeros((num_labels, dim), device=data.device) # float32
...
centroids[k] = cluster_k.mean(dim=0) # float64 mean, narrowed on assignment
intra_dists[k] = (cluster_k - centroids[k]).pow(2.0).sum(dim=1).sqrt().mean()
```

Writing a `float64` centroid into the `float32` buffer rounds it to about 7 significant digits. At `offset=1e7` the stored centroid is wrong by roughly `1e0`, while the true intra-cluster spread is roughly `0.3` — so `cluster_k - centroids[k]` is dominated by the rounding error rather than by the data.

`functional/clustering/calinski_harabasz_score.py`:

```python
between_cluster_dispersion = torch.tensor(0.0, device=data.device) # float32
within_cluster_dispersion = torch.tensor(0.0, device=data.device) # float32
```

Here only the running accumulation is narrowed — the per-cluster terms are still computed in the input dtype — so the damage is bounded at about `1e-7` relative rather than being catastrophic. It is still a silent `float32` ceiling on a `float64` input.

Both also return `torch.tensor(..., dtype=torch.float32)` explicitly from their degenerate branches, which would stay inconsistent even after the buffers are fixed.

### Expected behavior

The accumulators should be allocated in the input dtype, so `float64` in gives `float64` out and the computation is actually carried out at the requested precision — matching `dunn_index`.

### Environment

- TorchMetrics: `master` (commit 37a80809)
- PyTorch: 2.13.0+cpu, Python 3.13, Windows

### Related

Same class of problem as #3465 (`spearman_corrcoef` / `kendall_rank_corrcoef`) but a different mechanism — that one is integer division falling back to the default dtype, this one is untyped buffer allocation. Reporting separately since the fix and the blast radius are different.

I have a fix and regression tests ready and am happy to open a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read functional/clustering/davies_bouldin_score.py and functional/clustering/calinski_harabasz_score.py, then compare their dtype handling with dunn_index. Run the existing clustering tests and add regression coverage for float64 inputs, including off-origin data. Done means the affected metrics preserve the input dtype and return the expected values in degenerate cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.