Suggestion: Limit the size of the datacite.xml generated by Dataverse for datasets with many datafiles (thousands)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
Overview of the Suggestion
Limit the size of the datacite.xml generated by Dataverse for datasets with many datafiles (thousands) by only sending one size element (for the sum of all datafile sizes) and one unique format element for each datafile format that occurs, instead of sending a size and a format element for each datafile.
What kind of user is the suggestion intended for?
The administrator of the Dataverse installation.
What inspired this idea?
For the past few months, publication of datasets with thousands of files has been failing due to a 408 timeout error when uploading the datacite.xml to DataCite during the publication process. Also, the usefulness of having a size and format element for each datafile is questionable.
What existing behavior do you want changed?
Currently, the datacite.xml generated by Dataverse contains both a size as well as a format element for each file. The idea is to change this to one size element for the sum of all file sizes and one unique format element for each file format that occurs. This could be made configurable with a JVM option dataverse.datacite.xml.datafile-info with the following modes:
expanded: (default) the current behavior, one size and one format element for each datafilebrief: one size element for the sum of all datafile sizes and one unique format element for each datafile format that occursnone: no size or format elements at all
Note, the mode only takes effect on the next dataset version or when calling https://guides.dataverse.org/en/latest/admin/dataverses-datasets.html#update-metadata-for-a-published-dataset-at-the-pid-provider
Any brand new behavior do you want to add to Dataverse?
No, this is a modification to existing behavior.
Any open or closed issues related to this suggestion?
None that I am aware of.
Are you thinking about creating a pull request for this issue?
Yes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the code that generates datacite.xml and the publication path that uploads it to DataCite. Read how dataset datafile sizes and formats are currently emitted, then implement the requested dataverse.datacite.xml.datafile-info modes and verify that each mode affects the next dataset version or metadata update as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 54/100