IQSS / IQSS/dataverse

Feature Request: Setting Content-Length header for files being downloaded

Open
#10,927 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Feature
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

I would like to bring to your attention an issue regarding the Content-Length header. Currently, when accessing files through links such as: https://bonndata.uni-bonn.de/api/access/datafile/ID (as well as via persistentId), the Content-Length header is not being set.

This leads to problems with large file downloads (e.g., files over 20GB in our case), where only part of the file is downloaded without the download manager detecting the error.

I understand that using S3 is recommended for large files and likely avoids this issue, but I believe it would be beneficial to address this for file storage as well. Additionally, while I recognize that it may not be possible to calculate Content-Length when zipping files on the fly, this should be feasible for static files.

What kind of user is the feature intended for?
API User, Guest

What existing behavior do you want changed?
Adding Content-Length in response header

What do you think?

Thank you!
Sergej

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the /api/access/datafile/ID and persistentId download entry points described in the issue, then trace how responses are produced for static files versus files zipped on the fly. Done means static file downloads include a Content-Length response header while on-the-fly archives retain their existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.