NCAS-CMS / NCAS-CMS/cf-python

`Data.stats` edge case `AttributeError` w/ masked array

Open
#589 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

When Data.stats is applied to an array with a non-trivial mask (i.e. having at least one masked element), with both all=True and weights=0 set (for zero or Falsy weights arguments only, even though that doesn't really make sense to specify for any useful purpose but should regardless be handled elegantly in case someone tries it or ends up with that), it returns an obscure AttributeError, whereas with only one of those keywords set, or a non-masked array, it handles everything as it should, as shown in the snippet below.

This is an edge case which will probably not give any useful result due to the zero weighting for everything, but we should catch the error and give a more useful one regardless.

>>> import cf
>>> d = cf.Data([0, 1, 2], mask=[0, 1, 0])
>>> d.stats(all=True, weights=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sadie/cf-python/cf/data/data.py", line 9771, in stats
    return {op: val.array.item() for op, val in data_values.items()}
  File "/home/sadie/cf-python/cf/data/data.py", line 9771, in <dictcomp>
    return {op: val.array.item() for op, val in data_values.items()}
  File "/home/sadie/cf-python/cf/data/data.py", line 4688, in array
    array = self.compute().copy()
  File "/home/sadie/cf-python/cf/data/data.py", line 2368, in compute
    a.harden_mask()
  File "/home/sadie/anaconda3/envs/cf-env/lib/python3.8/site-packages/numpy/ma/core.py", line 3556, in harden_mask
    self._hardmask = True
  File "/home/sadie/anaconda3/envs/cf-env/lib/python3.8/site-packages/numpy/ma/core.py", line 6540, in __setattr__
    raise AttributeError(
AttributeError: attributes of masked are not writeable
>>> # But note that:
>>> d.stats(all=True)
{'minimum': 0, 'mean': 1.0, 'median': 1.0, 'maximum': 2, 'range': 2, 'mid_range': 1.0, 'standard_deviation': 1.0, 'root_mean_square': 1.4142135623730951, 'minimum_absolute_value': 0, 'maximum_absolute_value': 2, 'mean_absolute_value': 1.0, 'mean_of_upper_decile': 2.0, 'sum': 2, 'sum_of_squares': 4, 'variance': 1.0, 'sample_size': 2}
>>> d.stats(weights=0)
{'minimum': 0, 'mean': nan, 'median': 1.0, 'maximum': 2, 'range': 2, 'mid_range': 1.0, 'standard_deviation': 0.0, 'root_mean_square': nan, 'sample_size': 2}
>>> d.stats(all=True, weights=1)
{'minimum': 0, 'mean': 1.0, 'median': 1.0, 'maximum': 2, 'range': 2, 'mid_range': 1.0, 'standard_deviation': 1.0, 'root_mean_square': 1.4142135623730951, 'minimum_absolute_value': 0, 'maximum_absolute_value': 2, 'mean_absolute_value': 1.0, 'mean_of_upper_decile': 2.0, 'sum': 2, 'sum_of_squares': 4, 'variance': 1.0, 'sample_size': 2}
>>> # etc.
Environment

Snippet shown used:

$ cf.environment(paths=False)
Platform: Linux-4.15.0-54-generic-x86_64-with-glibc2.10
HDF5 library: 1.10.6
netcdf library: 4.8.0
udunits2 library: /home/sadie/anaconda3/envs/cf-env/lib/libudunits2.so.0
ESMF: 8.1.1
Python: 3.8.10
dask: 2023.1.0
netCDF4: 1.5.6
psutil: 5.9.0
packaging: 21.3
numpy: 1.22.2
scipy: 1.8.0
matplotlib: 3.4.3
cftime: 1.6.0
cfunits: 3.3.4
cfplot: 3.1.18
cfdm: 1.10.0.2
cf: 3.14.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cf/data/data.py at Data.stats, especially the return around line 9771, and trace the masked-array computation through the compute and array paths shown in the traceback. Reproduce the masked-array case with all=True and weights=0, then ensure it produces a useful error rather than the obscure AttributeError while the neighboring examples continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.