Inconsistent data between the harvesting server and client
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
After harvesting datasets we are finding differences between client and server. This mainly consider dates.
When I check api by calling /api/datasets/export?exporter=dataverse_json&persistentId=doi%3A10.57903/UJ/RH0J5L
I can see proper data in fields like lastUpdateTime and releaseTime.
However, the Dataverse UI shows the dates related to the harvesting moment.
Those wrong dates I can find in db by SELECT * FROM dvobject WHERE dtype = 'Dataset' AND authority = '10.xxx' AND identifier = 'xxx'
What is more sometimes even the title is wrong but with the same pattern: dataverse_json show correct values, but wrong one are stored in db and displayed to user.
This is confusing and should not happen. Problem is not related to dataset versions.
- When does this issue occur?
This is a bug related to harvesting and was noticed in the context of dates displayed to Harvester users.
When trying to fix by deleting harvesting client/server and reharvesting data from Dataverse collection - all datasets received one date on harvester: current date (as mentioned above: misleading dates are in dvobject as lastUpdateTime and releaseTime - when I correct them manually in db, then they are correctly displayed for end user)
-
Which page(s) does it occurs on?
Harvester pages visible for users. -
What happens?
Harvested data are incorrect.
- To whom does it occur (all users, curators, superusers)?
all
- What did you expect to happen?
harvesting client and server should display same data for users.
Which version of Dataverse are you using?
6.7.1 at this moment.
Screenshots:
dataverse_json:
displayed on harvester page:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the mismatch through /api/datasets/export?exporter=dataverse_json and the harvester pages, then compare those values with the corresponding dvobject row. Focus on the harvesting flow and the lastUpdateTime, releaseTime, and title fields, including a delete-and-reharvest cycle. Done means harvested client and server data match and the UI shows the source dates and title.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100