isce-framework / isce-framework/isce3

(CF Compliance) `valid_min` and `valid_max` Attribute Dtype Mismatch

Open
#304 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
239
Forks
90
Avg merge
13d 1h
Merged PRs (30d)
5

Description

Description: The valid_min and valid_max attributes have different data types than the datasets they describe. This is a related (but separate) issue from #297 .

CF Requirement (Section 2.5.1):

"The attributes valid_min, valid_max and valid_range ... are all of the same type as the variable to which they are attached."

— CF Conventions Version 1.7, Section 2.5.1: Missing Data

Source: http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html#_missing_data

Note: This issue primarily impacts RSLC and the five InSAR. (GSLC and GCOV resolved this issue via #213 ).

Impact:

  • Inconsistent behavior across different CF-compliant software
  • Tools may reject values that should be valid or accept invalid values

Fix Required: Ensure valid_min and valid_max use the same dtype as their parent dataset.

Common Patterns:

  1. float32 datasets with float64 valid_min/max (most common)

    • Geolocation/radarGrid metadata cubes
    • Affects: RSLC, RIFG, RUNW, ROFF, GUNW, GOFF
  2. uint32 datasets with int64 valid_min (no valid_max)

    • Mask datasets
    • Affects: RIFG, RUNW, ROFF, GUNW, GOFF
Occurrences by Product:

RSLC

File: NISAR_L1_PR_RSLC_001_149_D_080_2005_QPDH_A_20251003T130952_20251003T131016_X05016_N_P_J_001.h5

Dataset: /science/LSAR/RSLC/metadata/geolocationGrid/alongTrackUnitVectorX
  Dataset dtype: float32
  valid_min (actual): -1.0 (dtype: float64)
  valid_min (should be): np.float32(-1.0)
  valid_max (actual): 1.0 (dtype: float64)
  valid_max (should be): np.float32(1.0)

Other affected datasets (same pattern):
  - /science/LSAR/RSLC/metadata/geolocationGrid/alongTrackUnitVectorY
  - /science/LSAR/RSLC/metadata/geolocationGrid/elevationAngle (valid_min: 0.0, valid_max: 90.0)
  - /science/LSAR/RSLC/metadata/geolocationGrid/incidenceAngle (valid_min: 0.0, valid_max: 90.0)
  - /science/LSAR/RSLC/metadata/geolocationGrid/losUnitVectorX
  - /science/LSAR/RSLC/metadata/geolocationGrid/losUnitVectorY

RIFG

File: NISAR_L1_PR_RIFG_013_118_A_171_014_2000_SH_20260222T102514_20260222T102520_20260306T102515_20260306T102521_X05016_P_P_J_001.h5

Pattern: /science/LSAR/RIFG/metadata/geolocationGrid/*
  - alongTrackUnitVectorX, alongTrackUnitVectorY (float32 dataset, float64 valid_min/max)
  - elevationAngle, incidenceAngle (float32 dataset, float64 valid_min/max)
  - losUnitVectorX, losUnitVectorY (float32 dataset, float64 valid_min/max)

Additional valid_min only (no valid_max):
  - /science/LSAR/RIFG/grids/frequencyA/pixelOffsets/mask (uint32 dataset, int64 valid_min)
  - /science/LSAR/RIFG/grids/frequencyA/unwrappedPhase/mask (uint32 dataset, int64 valid_min)

RUNW

File: NISAR_L1_PR_RUNW_007_106_A_140_008_7700_SH_20251211T140849_20251211T140926_20251223T140850_20251223T140926_X05016_N_P_J_001.h5

Same pattern as RIFG:
Pattern: /science/LSAR/RUNW/metadata/geolocationGrid/* (6 datasets with both valid_min and valid_max)

Additional valid_min only:
  - /science/LSAR/RUNW/grids/frequencyA/pixelOffsets/mask (uint32 → int64)
  - /science/LSAR/RUNW/grids/frequencyA/unwrappedPhase/mask (uint32 → int64)

ROFF

File: NISAR_L1_PR_ROFF_007_106_A_140_008_7700_SH_20251211T140849_20251211T140926_20251223T140850_20251223T140926_X05016_N_P_J_001.h5

Pattern: /science/LSAR/ROFF/metadata/geolocationGrid/* (6 datasets with both valid_min and valid_max)
  - alongTrackUnitVectorX/Y, elevationAngle, incidenceAngle, losUnitVectorX/Y
  - All: float32 dataset with float64 valid_min/max

Additional valid_min only:
  - /science/LSAR/ROFF/grids/frequencyA/pixelOffsets/mask (uint32 → int64)

GUNW

File: NISAR_L2_PR_GUNW_001_062_A_019_006_2000_SH_20250927T114508_20250927T114542_20251126T114516_20251126T114545_X05016_P_F_J_001.h5

Pattern: /science/LSAR/GUNW/metadata/radarGrid/* (6 datasets with both valid_min and valid_max)
  - alongTrackUnitVectorX/Y, elevationAngle, incidenceAngle, losUnitVectorX/Y
  - All: float32 dataset with float64 valid_min/max

Additional valid_min only (uint32 datasets with int64 valid_min):
  - /science/LSAR/GUNW/grids/frequencyA/pixelOffsets/mask
  - /science/LSAR/GUNW/grids/frequencyA/unwrappedInterferogram/mask
  - /science/LSAR/GUNW/grids/frequencyA/wrappedInterferogram/mask
  (Similar for frequencyB)

GOFF

File: NISAR_L2_PR_GOFF_005_106_A_140_006_7700_SH_20251117T140848_20251117T140925_20251129T140849_20251129T140925_X05016_P_P_J_001.h5

Pattern: /science/LSAR/GOFF/metadata/radarGrid/* (6 datasets with both valid_min and valid_max)
  - alongTrackUnitVectorX/Y, elevationAngle, incidenceAngle, losUnitVectorX/Y
  - All: float32 dataset with float64 valid_min/max

Additional valid_min only:
  - /science/LSAR/GOFF/grids/frequencyA/pixelOffsets/mask (uint32 → int64)

GSLC and GCOV: CORRECT

GSLC and GCOV correctly uses matching dtypes for valid_min/max attributes.
Example from GSLC:

Dataset: /science/LSAR/GSLC/metadata/radarGrid/alongTrackUnitVectorX
  Dataset dtype: float32
  valid_min: -1.0 (dtype: float32) ✓ CORRECT
  valid_max: 1.0 (dtype: float32) ✓ CORRECT

cc: @hfattahi @bhawkins @xhuang-jpl @seongsujeong @gshiroma

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names sample HDF5 files and dataset paths but no source file or test; start by locating the RSLC, RIFG, RUNW, ROFF, GUNW, and GOFF product writers, then compare their attribute handling with GSLC/GCOV and #213. Check the affected geolocation/radarGrid and mask datasets. Done means valid_min and valid_max match each parent dataset's dtype across the listed products.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.