isce-framework / isce-framework/isce3
`rfiLikelihood` metadata exceeds documented [0,1] interval for tone-rank algorithm
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 239
- Forks
- 90
- Avg merge
- 13d 1h
- Merged PRs (30d)
- 5
Description
Summary
The rfiLikelihood metadata values need to be harmonized across RFI detection algorithms.
The tone-rank algorithm (introduced in #257) outputs ISR (Interference to Signal Ratio) values that can greatly exceed the documented [0,1] interval, while the ST-EVD algorithm correctly produces values within the expected range. (FDNF algorithm should also be considered.)
Problem
The rfiLikelihood metadata field in RSLC, GSLC, GCOV, and QA products is documented as having values in the interval [0,1], where:
- 0 = lowest RFI severity
- 1 = highest RFI severity
- NaN = RFI detection was skipped
However, actual product data contains values far outside this range.
This metadata value is computed by the RSLC workflow and populated into the RSLC HDF5. This metadata is then copied by the downstream workflows and populated into their respective HDF5 files. A fix in RSLC's workflow should resolve the issue in the downstream products as well.
Example
Product: NISAR_L1_PR_RSLC_012_097_D_106_2005_DHDH_A_20260208T225030_20260208T225100_X05016_N_F_J_001.h5
RFI Mitigation Algorithm Used: tone-rank (from /science/LSAR/RSLC/metadata/processingInformation/algorithms/rfiMitigation)
Description attribute:
Severity of radio frequency interference (RFI) contamination in the data. Value is in the interval [0,1], where 0: lowest severity, and 1: highest severity (or NaN if RFI detection was skipped)
Actual values in the product:
| Dataset Path | Value | Expected Range |
|---|---|---|
science/LSAR/RSLC/metadata/calibrationInformation/frequencyA/HH/rfiLikelihood |
259756.91 | [0, 1] |
science/LSAR/RSLC/metadata/calibrationInformation/frequencyA/HV/rfiLikelihood |
534179.69 | [0, 1] |
science/LSAR/RSLC/metadata/calibrationInformation/frequencyB/HH/rfiLikelihood |
2.95 | [0, 1] |
science/LSAR/RSLC/metadata/calibrationInformation/frequencyB/HV/rfiLikelihood |
5.28 | [0, 1] |
All values exceed the documented maximum of 1, with frequency A values exceeding it by 5-6 orders of magnitude.
Root Cause
Different RFI detection algorithms produce values on different scales:
-
ST-EVD algorithm: Generates RFI likelihood as the ratio of CPIs detected with RFI eigenvalues to total CPIs, producing values in [0,1] interval (focus.py:1162-1179)
-
FDNF (Frequency Domain Notch Filter) algorithm: Returns the fraction of pulses contaminated by RFI (
rfi_pulse_count_sum / (num_pulses * num_rng_blks)), producing values in [0,1] interval (rfi_freq_null.py:206) -
Tone-rank algorithm (introduced in #257): Assigns
rfi_likelihood = np.max(isr)whereisris the Interference-to-Signal Ratio array (focus.py:1199-1206). ISR values can be >> 1, which violates the documented [0,1] interval.
The tone-rank algorithm assigns rfi_likelihood = np.max(isr) at focus.py:1206, where isr is the Interference-to-Signal Ratio array. This ISR value is then written to the rfiLikelihood dataset with the [0,1] interval description at SLC.py:1072-1078.
The algorithms have not been synchronized to produce consistent output scales.
Impact
- Users cannot reliably interpret
rfiLikelihoodvalues - The metadata description is misleading
- Downstream processing that assumes [0,1] range may fail or produce incorrect results
- Products generated with different RFI algorithms are not comparable
cc: @hfattahi @bhawkins @hb6688 @Tyler-g-hudson @rad-eng-59 @gshiroma @seongsujeong
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in python/packages/nisar/workflows/focus.py at the ST-EVD, FDNF, and tone-rank rfi_likelihood calculations, then inspect python/packages/isce3/signal/rfi_freq_null.py and python/packages/nisar/products/writers/SLC.py. Determine how the algorithms should produce a consistent [0,1] value, including the documented metadata description, and verify that RSLC and downstream product metadata remain interpretable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100