nasa / nasa/Common-Metadata-Repository

Bug Report: XML Parsing is not working correctly

Open
#2,364 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Clojure
Stars
397
Forks
108
Avg merge
3d 14m
Merged PRs (30d)
13

Description

I work on the NOAA CMR program. We use repository https://git.services.nesdis.noaa.gov/dissemination/catalog/cmr-repos/dev-cmr-base, and periodically migrate changes from this repository to it, in order to capture changes you make into ours.

I recently migrated the changes in this repository up to tag CMR-1.287.0-r25.4.2 (previous migration point was CMR-1.277.0-r25.2.5). Prior to the migration I was able to ingest our sample NOAA ISO file into CMR without issue. Now that I have migrated the latest code I am no longer able to ingest that same ISO file into CMR.

At first it appeared that the issue was that the warnings I was seeing became errors, but as it turns out the UMM-C object being derived by the code after the migration varies from what is derived from the code prior to the migration.

I am attaching the UMM-C objects that were generated along with the ISO I ingested:
GOES_ABI_L1B_RAD.xml - the ingested ISO
prior-UMM-C.json - the object generated from the pre-migration code
after-UMM-C.json - the object generated from the after-migration code

The variances are found within the RelatedUrls. ArchiveAndDistributionInformation and Platforms fields of the UMM-C object. In the 1st 2 fields, the specific variance is on the Format field. In the last field the pre-migration object had 3 platforms while the after-migration object lacked any.

The xpaths used in our code to obtain the Format field are:

  • /gmi:MI_Metadata/gmd:distributionInfo/gmd:MD_Distribution/gmd:distributor/gmd:MD_Distributor/gmd:distributorFormat/gmd:MD_Format/gmd:name/gco:CharacterString
  • /gmi:MI_Metadata/gmd:distributionInfo/gmd:MD_Distribution/gmd:distributionFormat/gmd:MD_Format/gmd:name/gco:CharacterString

The xpaths used in our code to obtain the Platforms field are:

  • /gmi:MI_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords/gmd:MD_Keywords
  • /gmi:MI_Metadata/gmi:acquisitionInformation/gmi:MI_AcquisitionInformation/gmi:platform/gmi:MI_Platform, and within this xpath: gmi:instrument/gmi:MI_Instrument

after-UMM-C.json
GOES_ABI_L1B_RAD.xml
prior-UMM-C.json

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing GOES_ABI_L1B_RAD.xml with prior-UMM-C.json and after-UMM-C.json, focusing on the listed Format and platform xpaths. Trace the Clojure ingestion path that derives RelatedUrls, ArchiveAndDistributionInformation, and Platforms from those paths. Done means the migrated code produces the expected formats and three platforms for this sample.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
backend, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.