IQSS / IQSS/dataverse

Issues with Automatic Unpacking/Uploading of ZIP files containing many files.

Open
#10,591 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature: File Upload & Handling Type: Bug
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

What steps does it take to reproduce the issue?
Upload a ZIP file containing more than 1000 files.
Attempt to upload another file immediately after receiving the error message from the initial upload.

  • When does this issue occur?

During the upload of ZIP files with more than 1000 files.
When initiating subsequent uploads before the deletion process of the initial 1000 files is completed.

  • Which page(s) does it occurs on?
    Dataset upload page

  • What happens?
    When a ZIP file containing more than 1000 files is uploaded, the extraction process stops after the first 1000 files with an error message.
    The 1000 files are then deleted, but users can start new uploads before the deletion process is finished. This can lead to files with the same name being present, causing the system to append a “-2” to the new file names.
    This results in inconsistencies in documentation, errors during software compilation, and other issues.

  • To whom does it occur (all users, curators, superusers)?
    All users - uploaders

  • What did you expect to happen?
    The system should check the number of files in the ZIP before attempting to extract and reject the upload if it exceeds the limit.
    After an error, the system should wait until all files are deleted before allowing new uploads.
    The system maybe should not automatically rename files upon encountering a name conflict.

Which version of Dataverse are you using?

6.2

Any related open or closed issues to this bug report?

There are several issues mentioning ZIP uploads, but not this behavior exactly.

Screenshots:

Additional suggestions
Introduce a checkbox allowing users to choose whether to unpack ZIP, TGZ archives, etc.
Add a warning message on the upload field indicating that archives will be unpacked, and mention the .ZIP.ZIP workaround.

No matter the issue, screenshots are always welcome.

To add a screenshot, please use one of the following formats and/or methods described here:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the Dataset upload page flow and trace ZIP extraction, the 1,000-file limit, cleanup after a failed upload, and filename conflict handling. Reproduce an upload containing more than 1,000 files, then start another upload during cleanup. Done means oversized archives are rejected safely and subsequent uploads cannot observe leftover files or receive unintended renamed duplicates.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.