Small temporary upload files are not automatically deleted after upload
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
What steps does it take to reproduce the issue?
When a file is uploaded, there are usually some upload files left in the system, in two spots.
- dataverse.files.file.directory
- /payara6/glassfish/domains/domain1/uploads
I think it's related to this issue, but not sure why it's not fixed in our setup: https://github.com/IQSS/dataverse/issues/10667
-
When does this issue occur?
This issue occurs after a file is uploaded via the UI -
Which page(s) does it occurs on?
File upload through the UI. -
What happens?
Files are left behind in: dataverse.files.file.directory. Here's a small sample:
-rw-r--r--. 1 dataverse dataverse 831213413 Jun 1 15:39 19e83d716d8-f6e08e442fd9
-rw-r--r--. 1 dataverse dataverse 838353413 Jun 1 15:39 19e83d75d03-bcba11fda004
-rw-r--r--. 1 dataverse dataverse 835973455 Jun 1 15:40 19e83d7a49f-4d11683f60bc
-rw-r--r--. 1 dataverse dataverse 10036 Jun 1 19:02 tempIngestSourceFile14944432732260873865.tmp
-rw-r--r--. 1 dataverse dataverse 255 Jun 1 19:02 firstpass-2728339230294518051.tab
-rw-r--r--. 1 dataverse dataverse 14065 Jun 1 19:02 tempIngestSourceFile674369604737836795.tmp
-rw-r--r--. 1 dataverse dataverse 99 Jun 1 19:02 firstpass-11771889047601741684.tab
-rw-r--r--. 1 dataverse dataverse 14853 Jun 1 19:02 tempIngestSourceFile16848639504657575746.tmp
-rw-r--r--. 1 dataverse dataverse 153 Jun 1 19:02 firstpass-1044911828866616382.tab
-rw-r--r--. 1 dataverse dataverse 15074 Jun 1 19:02 tempIngestSourceFile479945955968724125.tmp
-rw-r--r--. 1 dataverse dataverse 232 Jun 1 19:02 firstpass-12198861583407135376.tab
Sometimes these files are larger, depending on the upload.
As well as in: /payara6/glassfish/domains/domain1/uploads:
-rw-r--r--. 1 dataverse dataverse 17 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034335.tmp
-rw-r--r--. 1 dataverse dataverse 44 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034332.tmp
-rw-r--r--. 1 dataverse dataverse 34 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034329.tmp
-rw-r--r--. 1 dataverse dataverse 47 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034326.tmp
-rw-r--r--. 1 dataverse dataverse 22 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034350.tmp
-rw-r--r--. 1 dataverse dataverse 22 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034349.tmp
-rw-r--r--. 1 dataverse dataverse 45 Jun 1 19:02 upload_5df8c250_19e698dd064__7feb_00034348.tmp
These do not automatically get cleaned up by the code at all, though I thought this was fixed by https://github.com/IQSS/dataverse/pull/10867
-
To whom does it occur (all users, curators, superusers)?
All users, I think. Other installs have this issue as well, post 6.4. #troubleshooting > Uploads directory content not clearing after 6.2 upgrade @ 💬 -
What did you expect to happen?
I expected the files to disappear when the upload/conversion was done. Instead they stick around.
Which version of Dataverse are you using?
6.6
Any related open or closed issues to this bug report?
https://github.com/IQSS/dataverse/pull/10867
Screenshots:
No matter the issue, screenshots are always welcome.
To add a screenshot, please use one of the following formats and/or methods described here:
Are you thinking about creating a pull request for this issue?
Help is always welcome, is this bug something you or your organization plan to fix?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing a UI upload in Dataverse 6.6 and inspect the two reported locations: dataverse.files.file.directory and /payara6/glassfish/domains/domain1/uploads. Read related issue #10667 and pull request #10867 to understand the existing cleanup behavior. Done means temporary upload, ingest, and conversion files are removed from both locations after processing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100