Need to enforce the ZipDownloadLimit in the access API consistently
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
When the total dataset storage size exceeds the ZipDownloadLimit, we don't offer the "download all" option on the dataset page. Showing the message "The dataset is too large to download. Please select the files you need from the files table."
However, if the user goes directly to the download-all API (/api/access/dataset/...), the limit is not enforced when the external zipper option is enabled. I.e., the api will happily try to redirect to the zipper even if it's a terabyte-size dataset - which is almost certainly not going to work.
We need to enforce the limit there the same way as when the zipping is done internally. I'm assuming we simply forgot to add the check there. (The initial, ill-conceived plan was to allow any size multi-file downloads with the external zipper; once we realized that was a mistake, we addressed it by enforcing the limit on the dataset page level, but not in the api).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the /api/access/dataset/... download-all API entry point and compare its external-zipper path with the internal zipping path. Confirm how ZipDownloadLimit is checked for the dataset page, then ensure the API rejects oversized datasets consistently instead of redirecting to the zipper. Done means the limit is enforced for both paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100