IQSS / IQSS/dataverse

Suggestion: Limit the size of the datacite.xml generated by Dataverse for datasets with many datafiles (thousands)

Open
#12,638 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Suggestion
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

Overview of the Suggestion
Limit the size of the datacite.xml generated by Dataverse for datasets with many datafiles (thousands) by only sending one size element (for the sum of all datafile sizes) and one unique format element for each datafile format that occurs, instead of sending a size and a format element for each datafile.

What kind of user is the suggestion intended for?
The administrator of the Dataverse installation.

What inspired this idea?
For the past few months, publication of datasets with thousands of files has been failing due to a 408 timeout error when uploading the datacite.xml to DataCite during the publication process. Also, the usefulness of having a size and format element for each datafile is questionable.

What existing behavior do you want changed?
Currently, the datacite.xml generated by Dataverse contains both a size as well as a format element for each file. The idea is to change this to one size element for the sum of all file sizes and one unique format element for each file format that occurs. This could be made configurable with a JVM option dataverse.datacite.xml.datafile-info with the following modes:

  • expanded: (default) the current behavior, one size and one format element for each datafile
  • brief: one size element for the sum of all datafile sizes and one unique format element for each datafile format that occurs
  • none: no size or format elements at all

Note, the mode only takes effect on the next dataset version or when calling https://guides.dataverse.org/en/latest/admin/dataverses-datasets.html#update-metadata-for-a-published-dataset-at-the-pid-provider

Any brand new behavior do you want to add to Dataverse?
No, this is a modification to existing behavior.

Any open or closed issues related to this suggestion?
None that I am aware of.

Are you thinking about creating a pull request for this issue?
Yes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the code that generates datacite.xml and the publication path that uploads it to DataCite. Read how dataset datafile sizes and formats are currently emitted, then implement the requested dataverse.datacite.xml.datafile-info modes and verify that each mode affects the next dataset version or metadata update as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.