NCAS-CMS / NCAS-CMS/cf-python

Configurably truncated representation when FieldList is very long

Open
#772 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

API review (4.0.0) enhancement
Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

In str and repr calls on a FieldList, we print a one-line view of all fields present in the list, which is generally useful and sensible, however when the FieldList is suitably long this can seem like far too much information that it becomes spammy and it becomes hard to access previous terminal or interactive Python calls and output. For example, working with WRF data I get >10,000 field representations spat out as a listing.

A nice feature would be to truncate this listing with an ellipsis (or otherwise, but that seems standard and sensible e.g. as used to truncate numpy arrays), when the list is over N items long for some sensible N:

[<CF Field: ncvar%ACGRDFLX(ncdim%Time(1), ncdim%south_north(179), ncdim%west_east(139)) J m-2>,
 <CF Field: ncvar%ACGRDFLX(ncdim%Time(1), ncdim%south_north(179), ncdim%west_east(139)) J m-2>,
 <CF Field: ncvar%ACGRDFLX(ncdim%Time(1), ncdim%south_north(179), ncdim%west_east(139)) J m-2>,
...
 <CF Field: ncvar%ZS(ncdim%Time(1), ncdim%soil_layers_stag(4)) m>,
 <CF Field: ncvar%ZS(ncdim%Time(1), ncdim%soil_layers_stag(4)) m>,
 <CF Field: ncvar%ZS(ncdim%Time(1), ncdim%soil_layers_stag(4)) m>]

and perhaps to and also add a note of the full length to the start of the reporesentation to indicate how many fields have been subsumed into the ellipsis, e.g. to include a first line to the above such as this, or similar, where N is the length:

CF FieldList (N):
[ ... ]
Configurability

NumPy and others support configurability on the truncation threshold (point at which the representation gets summarised and not shown fully) via methods such as numpy.set_printoptions with a threshold argument, so ideally we can also allow the user to configure this for our FieldList (and maybe otherwise, with other truncation on aspects such as data array views?). I suggest adding another setting under cf.configuration() called print_threshold or similar, taking an integer as per the numpy threshold parameter.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the FieldList str and repr implementations and the cf.configuration() entry point mentioned in the issue. Review existing representation tests, then define the truncation threshold and configuration behavior; done means long FieldList representations are abbreviated while shorter ones remain unchanged and the threshold can be configured.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.