IQSS / IQSS/dataverse

cleanStorage can delete an upload that is still in progress

Open
#12,689 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Bug
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

What steps does it take to reproduce the issue?

  • When does this issue occur?

When cleanStorage runs on a dataset that has an upload which has finished transferring but has not been saved yet, and whenever the endpoint is reached without anyone intending to delete anything.

Upload a file to a dataset with direct upload and leave the page without saving, so the object is in the dataset's storage but no DataFile row exists for it yet. Call GET /api/datasets/{id}/cleanStorage on that dataset. The object is deleted, because it looks the same as an abandoned upload.

The call is a GET and dryrun defaults to false, so it also deletes when nobody asked it to. Browser prefetch, a link preview in a chat or mail client, a crawler, or opening the URL again from history is enough.

  • Which page(s) does it occurs on?

API only: /api/datasets/{id}/cleanStorage.

  • What happens?

Storage objects are deleted. Files that belong to an upload in progress are deleted along with the abandoned ones, and a plain GET with no query parameters deletes rather than reports.

A user without EditDataset on the dataset gets 500 instead of 403.

  • To whom does it occur (all users, curators, superusers)?

Anyone with EditDataset on the dataset. The upload that gets deleted can belong to a different user working on the same dataset.

  • What did you expect to happen?

Cleanup should remove abandoned files only, never a file that is still being worked on. A request that deletes should not be a GET, and omitting dryrun should report rather than delete.

Which version of Dataverse are you using?

Present in 6.10 and 6.11, and on develop.

Any related open or closed issues to this bug report?

None found.

Screenshots:

Not applicable, API only.

Are you thinking about creating a pull request for this issue?

Yes, a fix is ready: PUT instead of GET, dryrun defaulting to true, a configurable minimum age so recently modified objects are left alone, and 403 for the permission failure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the implementation of GET /api/datasets/{id}/cleanStorage and trace its cleanup and permission handling. Verify that deletion requires the intended method, dryrun defaults to reporting, recently modified upload objects are preserved, and missing EditDataset permission returns 403 rather than 500.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.