NCAS-CMS / NCAS-CMS/cf-python

Warning with ineffectual and invalid collapses

Open
#110 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

If a user tries to perform a collapse on an axes which is already singular, depending on the nature of the collapse method this may change the data and metadata as expected e.g. for 'sum_of_squares', but for most (at least, unweighted) collapse methods it, very reasonably, does not change anything, including not adding a call method indicating the collapse type, because the operation would not have an effect or be invalid (see examples below).

I think we should provide feedback to indicate to the user if a collapse has not had effect, since (say if they attempt such a collapse when not realising the axis/es size(s) is/are already one) they may be relying on metadata changes that would be made if the collapse had in fact been applied, such as the fact the units have changed appropriately, or an appropriate cell method has been added, etc. It certainly has the potential to cause confusion when collapses promise to make such changes.

The reason I am not implementing such feedback off the bat, as I doubt that is controversial, is that I am not sure of the best form of feedback, notably:

  • should we always use a logging message, or could it even warrant an exception for the 'invalid' case (number 3 below)?
  • what level of logging message would be most appropriate (one of 'info' or 'warning' I would think), and should this differ based on case (as below)?
Illustrative cases & examples

As far as I can see there are three cases for a size-one collapse, where case (1) is perfectly fine, but cases (2) and (3) warrant some form of user feedback and (3) may warrant a stronger form of warning, or even an Exception of some sort:

(All example code snippets operating on the following field f:

>>> import cf
>>> f=cf.read('~/Downloads/cfplot_data/ggap.nc')[1]
>>> f
<CF Field: eastward_wind(time(1), pressure(23), latitude(160), longitude(320)) m s**-1>
>>> print(f.data)
[[[[-13.383878707885742, ..., 1.6122403144836426]]]] m s**-1

)

  1. It has an effect as with a collapse of axes with non-singular size, e.g. 'sum_of_squares':

    >>> i = f.collapse('sum_of_squares', axes='T')
    >>> print(i.cell_methods)
    Constructs:
    {'cellmethod0': <CF CellMethod: domainaxis0: sum_of_squares>}
    >>> print(i.data)
    [[[[179.12820926739732, ..., 2.5993188316463147]]]] Gy
    
  2. It has no effect because it would make no difference to the field, e.g. 'maximum' (note no cell methods are added):

    >>> j = f.collapse('maximum', axes='T')
    >>> print(j.cell_methods)
    Constructs:
    {}
    >>> print(j.data)
    [[[[-13.383878707885742, ..., 1.6122403144836426]]]] m s**-1
    
  3. It has no effect because it is not possible, e.g. an unweighted unbiased variance collapse, given that the denominator of the corresponding calculation becomes (1 - 1) = 0:

    >>> k = f.collapse('variance', axes='T')
    >>> print(k.cell_methods)
    Constructs:
    {}
    >>> print(k.data)
    [[[[-13.383878707885742, ..., 1.6122403144836426]]]] m s**-1
    

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the Field.collapse entry point shown in the examples and trace how size-one axes are handled for maximum, variance, and sum_of_squares. Decide and document the feedback for effective, no-op, and invalid collapses, then add coverage for the three illustrated cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.