IQSS / IQSS/dataverse

Need to enforce the ZipDownloadLimit in the access API consistently

Open
#8,895 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

When the total dataset storage size exceeds the ZipDownloadLimit, we don't offer the "download all" option on the dataset page. Showing the message "The dataset is too large to download. Please select the files you need from the files table."

However, if the user goes directly to the download-all API (/api/access/dataset/...), the limit is not enforced when the external zipper option is enabled. I.e., the api will happily try to redirect to the zipper even if it's a terabyte-size dataset - which is almost certainly not going to work.

We need to enforce the limit there the same way as when the zipping is done internally. I'm assuming we simply forgot to add the check there. (The initial, ill-conceived plan was to allow any size multi-file downloads with the external zipper; once we realized that was a mistake, we addressed it by enforcing the limit on the dataset page level, but not in the api).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the /api/access/dataset/... download-all API entry point and compare its external-zipper path with the internal zipping path. Confirm how ZipDownloadLimit is checked for the dataset page, then ensure the API rejects oversized datasets consistently instead of redirecting to the zipper. Done means the limit is enforced for both paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.