Indexing race conditions for new dataset versions
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
What steps does it take to reproduce the issue?
- When does this issue occur?
When a new dataset version is created and immediately indexed, there is a race condition where the indexing process starts before the transaction that created the dataset version has fully committed to the database. Consequently, the search index may contain entries with a missing datasetVersionId.
This causes server errors when retrieving those drafts from the Solr index later, e.g. when accessing the search API:
[#|2026-04-28T13:40:41.705+0000|SEVERE|Payara 6.2025.10||_ThreadID=321;_ThreadName=http-thread-pool::http-listener-1(12);_TimeMillis=1777383641705;_LevelValue=1000;|
java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because "datasetVersionId" is null
at edu.harvard.iq.dataverse.search.SolrSearchServiceBean.solrDocumentToSolrSearchResult(SolrSearchServiceBean.java:263)
at edu.harvard.iq.dataverse.search.SolrSearchServiceBean.search(SolrSearchServiceBean.java:717)
This has occurred mainly and regularly when creating a lot of new dataset versions at once via an automated script.
- Which page(s) does it occurs on?
The issue affects pages that display search results, including the main search page, collection (dataverse) pages, and Search API calls.
-
What happens?
When search results are processed, the code attempts to use a null or missingdatasetVersionId. For search API calls, this led to 500 errors (Internal Server Error). -
To whom does it occur (all users, curators, superusers)?
This can occur to any user performing a search or viewing a collection that includes recently created or updated datasets that were affected by the race condition.
- What did you expect to happen?
Search results should always display correctly. Ideally, the indexer should only run once the data is fully committed.
Which version of Dataverse are you using?
6.9
Any related open or closed issues to this bug report?
Not aware of any.
Screenshots:
/
Are you thinking about creating a pull request for this issue?
Help is always welcome, is this bug something you or your organization plan to fix?
Yes, we have implemented a potential fix and will open a PR asap.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with SolrSearchServiceBean.java at solrDocumentToSolrSearchResult (line 263) and search (line 717), then trace how indexing is triggered after dataset-version creation. Reproduce the bulk dataset-version scenario and verify that search pages and Search API calls no longer fail or contain entries with a missing datasetVersionId.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100