NCAS-CMS / NCAS-CMS/cf-python

`pyfive` spams test suite STDOUT

Open
#928 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

When run as part of the full test suite, test_DSG_contiguous displays a lot of high-verbosity output from pyfive that spams the test suite, though when run standalone (python test_dsg.py) it doesn't. Partial output (curtailed from full STDOUT which shows thousands of lines) is as follows:

$ python run_tests.py
...
...
test_DSG_contiguous (test_dsg.DSGTest.test_DSG_contiguous) ... [pyfive] Obtained 3(3+0) attributes from offset 88 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Obtained 4(4+0) attributes from offset 132 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Accessing object '/obs' with link target 685 (lazy access: False)
[pyfive] Obtained 4(4+0) attributes from offset 132 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Accessing object '/row_size' with link target 1155 (lazy access: False)
[pyfive] Obtained 5(5+0) attributes from offset 126 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Accessing object '/bounds' with link target 8192 (lazy access: False)
[pyfive] Obtained 4(4+0) attributes from offset 132 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Accessing object '/time_bounds' with link target 8661 (lazy access: False)
[pyfive] Obtained 3(3+0) attributes from offset 262 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(9707, 9707); elapsed=0ms
[pyfive] Accessing object '/time' with link target 12323 (lazy access: False)
[pyfive] Obtained 7(7+0) attributes from offset 287 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(13203, 13203); elapsed=0ms
[pyfive] Accessing object '/lat' with link target 15299 (lazy access: False)
[pyfive] Obtained 6(6+0) attributes from offset 132 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Accessing object '/lon_bounds' with link target 16014 (lazy access: False)
[pyfive] Obtained 3(3+0) attributes from offset 148 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Accessing object '/lon' with link target 16607 (lazy access: False)
[pyfive] Obtained 7(7+0) attributes from offset 134 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Accessing object '/alt' with link target 18491 (lazy access: False)
[pyfive] Obtained 8(8+0) attributes from offset 134 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Accessing object '/station_name' with link target 19116 (lazy access: False)
[pyfive] Obtained 5(5+0) attributes from offset 135 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(19592, 19592); elapsed=0ms
[pyfive] Accessing object '/station_info' with link target 21688 (lazy access: False)
[pyfive] Obtained 4(4+0) attributes from offset 126 (fh_id=140712493448032 type=BufferedReader) in 0.0001s
[pyfive] Accessing object '/humidity' with link target 22117 (lazy access: False)
[pyfive] Obtained 6(6+0) attributes from offset 194 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(22851, 22851); elapsed=0ms
[pyfive] Accessing object '/temp' with link target 24947 (lazy access: False)
[pyfive] Obtained 7(7+0) attributes from offset 194 (fh_id=140712493448032 type=BufferedReader) in 0.0002s
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(25733, 25733); elapsed=0ms
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Accessing object '/obs' with link target 685 (lazy access: False)
[pyfive] Accessing object '/obs' with link target 685 (lazy access: False)
[pyfive] Accessing object '/obs' with link target 685 (lazy access: False)
[pyfive] Accessing object '/bounds' with link target 8192 (lazy access: False)
[pyfive] Accessing object '/bounds' with link target 8192 (lazy access: False)
[pyfive] Accessing object '/bounds' with link target 8192 (lazy access: False)
Reading netCDF file: /home/slb93/git-repos/cf-python/cf/test/tmp55h9odyr_test_dsg.nc
...
[pyfive] Building chunk index (pyfive version=1.1.1)
[pyfive] Chunk index built: btree range=(25733, 25733); elapsed=0ms
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Accessing object '/station' with link target 331 (lazy access: False)
[pyfive] Accessing object '/station_info' with link target 21688 (lazy access: False)
    Referenced netCDF variables:alt
        lat
        lon
        lon_bounds
        station_info
        station_name
        time
        time_bounds
        row_size
    Unreferenced netCDF variables:
        humidity
        temp
Unaggregatable 'specific_humidity' has been output: <CF AuxiliaryCoordinate: cf_role=timeseries_id(4) > has no identity or no data
Unaggregatable 'air_temperature' has been output: <CF AuxiliaryCoordinate: cf_role=timeseries_id(4) > has no identity or no data
Read 2 fields from 1 dataset
ok

So seems like some verbosity setting/changing interaction might be at play and this might not hit users and only the test suite, but we should investigate this is case it could emerge in some user-facing way, anyhow (with low priority).

(A quick/skim read of the pyfive docs isn't indicating anything that might toggle verbosity as an obvious cause of this.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running test_dsg.py standalone and through run_tests.py, focusing on test_DSG_contiguous and the pyfive output shown in the issue. Compare the verbosity behavior in both paths and consult the linked pyfive documentation. Done means the full test suite no longer emits the unexpected high-volume output, without changing the test result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.