pytroll / pytroll/satpy

Wrong units for DNB-based composites

Open
#1,985 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
335
Avg merge
1d 18h
Merged PRs (30d)
10

Description

Describe the bug

There are several DNB-based single-channel composites built into Satpy: adaptive_dnb, histogram_dnb, dynamic_dnb, and hncc_dnb. They all do more or less complex operations on the calibrated radiances. I believe those are all intended for visualisation purposes and not for further processing.

Those more or less complex operations have impacts on the unit of the data. The resulting value can no longer be directly interpreted as a radiance. Perhaps the correct units include a sqrt, a log, or no unit can be assigned at all.

However, in reality, the hncc_dnb composite claims to present radiances in W m¯² sr¯¹, whereas the histogram_dnb, adaptive_dnb, and dynamic_dnb all claim to present radiances in units 1. Radiance cannot have unit 1, and I think the hncc_dnb composite takes the square root of the data, so in all cases the units are wrong.

To Reproduce

from satpy import Scene
from glob import glob
#from satpy.utils import debug_on; debug_on()
filenames = glob("/media/nas/x21308/scratch/VIIRS/npp_20220119_0132_53000/*DNB*.h5")
sc = Scene(filenames=filenames, reader=["viirs_sdr"])
names = ["DNB", "hncc_dnb", "histogram_dnb", "adaptive_dnb", "dynamic_dnb"]
sc.load(names)
for name in names:
    print(name, sc[name].attrs["calibration"], sc[name].attrs["units"])

Expected behavior

I expect that DNB has units of W m¯² sr¯¹, but the others should all have different units or no units at all.

Actual results

The units are expected for DNB but unexpected for all others.

DNB radiance W m-2 sr-1
hncc_dnb radiance W m-2 sr-1
histogram_dnb radiance 1
adaptive_dnb radiance 1
dynamic_dnb radiance 1

Environment Info:

  • OS: openSUSE 15.3
  • Satpy Version: v0.33.1-39-gcccbe06f

Additional context

The unit 1 causes problems in pyninjotiff, which tries and fails to convert it to % or N/A, depending on what I configure. As a consequence, pyninjotiff cannot write the DNB composites.

The wrong units also appear to affect composites. When I load true_color, the metadata also state calibration: 'radiance' and units: '%'. I'm not sure why in this case pyninjotiff has no problems writing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Scene-loading example and comparing the calibration and units attributes for DNB, hncc_dnb, histogram_dnb, adaptive_dnb, and dynamic_dnb. Trace the implementations of those composite entry points and determine how their transformations affect metadata; done means the reported units no longer claim radiance incorrectly and the composites can be written without the described unit-conversion failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.