IQSS / IQSS/dataverse

Refactor and clean up the Access (download) API

Open
#7,721 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

This is a "technical debt" and legacy issue.

Access is one of the oldest APIs in the application, originally implemented in the context of DVN 3. There is a lot in the code that is archaic and cumbersome and hard to work with.
Prime examples:

  • Unlike most of our other APIs, file downloads need to serve potentially massive amounts of data. So the content cannot be returned as a buffer or a string, and needs to be streamed. To implement streaming, in most places we are still relying on the code written for early versions of Jersey REST library; that requires providing standalone MessageBodyWriter classes (DownloadInstance.java, DownloadInstanceWriter.java etc.) It would be cleaner to switch to using javax.ws.rs.core.StreamingOutput (/api/access/datafiles, download-multiple-files-zipped, a newer API, uses it already).
  • There's a system of OptionalAccessService classes, meant to support extra download options - extra formats/conversions and such. It was meant to be dynamic/possibly plugin-driven. It became hard-coded in the original Dataverse 4.0. And by now it only makes the code more confusing and difficult to work with.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the Access download API entry points, including /api/access/datafiles, and the legacy DownloadInstance.java and DownloadInstanceWriter.java classes. Compare their streaming approach with the existing StreamingOutput implementation, then trace the OptionalAccessService classes and their hard-coded uses. Done requires a defined, tested cleanup scope for both download streaming and optional access services.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.