IQSS / IQSS/dataverse

Harvesting: Fails OAI validation on several tests.

Open
#8,815 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature: Harvesting Type: Bug
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

Reported in RT, 322283, OAI Validation (step 2): https://demo.dataverse.org/oai
http://www.openarchives.org/Register/ValidateSite?log=YEI0NF6C

The basic validation, which tests IDENTIFY works at: https://www.openarchives.org/Register/ValidateSite
Their test log above tests the entire suite and shows two failures:
Checking exception handling (errors)
REQUEST https://demo.dataverse.org/oai?junk GET
WARN Bad HTTP status code from server: 500
FAIL Can't parse malformed response.
REQUEST https://demo.dataverse.org/oai?verb=junk GET
WARN Bad HTTP status code from server: 500
FAIL Can't parse malformed response.

Summary - failure
Uses https URIs (not specified in protocol)
Total tests passed: 35
Total warnings: 3
Total error count: 2
Validation status: FAILED

I have requested the steps to run this more advanced test to make sure we are testing the same way. Additionally, I've found this tester and you can select subsets of operations to test. https://validator.oaipmh.com/ Using this one, it works for most but fails the following two subsets:

  1. ListMetadataFormats (https://validator.oaipmh.com/#ListMetadataFormats)
    -Invalid schema URL http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd in Datacitehttp://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsdhttp://datacite.org/schema/kernel-3
    -Schema http://schema.datacite.org/meta/kernel-4.1/metadata.xsd not found!
    -Invalid schema URL JSON schema pending in dataverse_jsonJSON schema pendingCustom Dataverse metadata in JSON format (Dataverse4 to Dataverse4 harvesting only)
  2. ListRecords DATAVERSE_JSON
    -Line 1, Column 0, Element '{http://www.openarchives.org/OAI/2.0/}metadata', attribute 'directApiCall': The attribute 'directApiCall' is not allowed.
    -Line 1, Column 0, Element '{http://www.openarchives.org/OAI/2.0/}metadata': Missing child element(s). Expected is ( ##other{http://www.openarchives.org/OAI/2.0/}* ).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failures against https://demo.dataverse.org/oai using the Open Archives validation tools linked in the issue, including malformed requests, ListMetadataFormats, and ListRecords DATAVERSE_JSON. Done means the reported OAI validation errors no longer occur and the validator reports success for these cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.