🐛[BUG]: Variance in metrics.general.ensemble_metrics ignores the dim argument
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.3k
- Forks
- 787
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 27
Description
Version
main at ff5d19d08123de47ca446caed1d70a225d540184
On which installation method(s) does this occur?
Source
Describe the issue
physicsnemo.metrics.general.ensemble_metrics.Variance.__call__ accepts a dim argument but two lines inside it still assume the ensemble dimension is the leading one.
- The sample count is taken from
inputs.shape[0]instead ofinputs.shape[dim].Mean.__call__a few lines above usesinputs.shape[dim]. - The centered sum of squares is
torch.sum((inputs - self.sum / self.n) ** 2, dim=dim).self.sumhas the reduced shape, so the subtraction only broadcasts whendimis the leading dimension.
As a result, for any dim other than 0 the call either raises a broadcasting RuntimeError, or, when the size of dim happens to equal the size of the leading dimension, it runs through and silently returns a wrong variance. The silent case is the worrying one because it is the shape you get from a square ensemble by time grid.
_update_var has the same broadcasting problem for a batch_dim other than 0 because it computes inputs - temp_sum / temp_n with the already reduced temp_sum.
The existing test_means_var only exercises dim=0 and is skipped when CUDA is not available, so CI never saw this.
I expected Variance(...)(x, dim=d) to match torch.var(x, dim=d) for every d, the same way Mean(...)(x, dim=d) matches torch.mean(x, dim=d).
Minimum reproducible example
import torch
import physicsnemo.metrics.general.ensemble_metrics as em
# Broadcasting error
x = torch.randn(4, 6, 5)
em.Variance((4, 5))(x, dim=1)
# RuntimeError: The size of tensor a (6) must match the size of tensor b (4) at non-singleton dimension 1
# Silent wrong result when the sizes happen to match
x = torch.randn(6, 6, 5)
var = em.Variance((6, 5))(x, dim=1)
print(torch.allclose(var, torch.var(x, dim=1)))
# False
# Mean is fine on the same input
print(torch.allclose(em.Mean((6, 5))(x, dim=1), torch.mean(x, dim=1)))
# True
Relevant log output
RuntimeError: The size of tensor a (6) must match the size of tensor b (4) at non-singleton dimension 1
Environment details
Bare-metal, CPU only, Python 3.12, torch CPU wheel, physicsnemo installed from source with pip install -e .
I have a fix ready with a CPU test that covers dim 1, 2 and -1 for both the error case and the silent case, and I will open a PR that references this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in physicsnemo.metrics.general.ensemble_metrics at Variance.call, comparing its dimension handling with Mean.call; also inspect _update_var for the related batch_dim case. Run the existing test_means_var, then extend CPU coverage for dimensions 1, 2, and -1, including mismatched and equal leading-dimension sizes. Done means variance matches torch.var for each covered dimension.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100