Problem uploading large files

Nobody has claimed this yet.

What steps does it take to reproduce the issue?

Install Dataverse using the dataverse-ansible playbook
Create an empty dataset
Create a large file with random data: head -c 1G /dev/urandom > sample_1GB.txt
Upload it using python-dvuploader: dvuploader sample_1GB.txt --pid https://doi.org/10.XXXXXX --api-token XXXXXXXXXXXXXXXXX --dataverse-url http://dataverse-test

When does this issue occur?

Uploading data using python-dvuploader. This could be a bug from python-dvuploader. I put this bug in here because I found a possibly related issue in this repo.

What happens?

After some time, the upload stalls at 100%, the process never ends but at the dataset start to appear copies of the file with variations of the name

If you upload it with using the web UI, the file is uploaded correctly. If you upload it using the REST API, the file is uploaded but the process ends with this error page:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at 
 root@localhost to inform them of the time this error occurred,
 and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>

To whom does it occur (all users, curators, superusers)?

All users that can upload data

What did you expect to happen?

The file being uploaded once.

Which version of Dataverse are you using?
Dataverse 6.9 and 6.10.1

Any related open or closed issues to this bug report?

Screenshots:

Logs:
Payara log:
https://gist.github.com/cperezmig-dipc/a53a3b5da46a3567af523714be103155

Apache logs:
https://gist.github.com/cperezmig-dipc/28d316f6bce4b677b49c9cba4e21762d

Related discussion:

#troubleshooting > Problem uploading large file with python dvuploader @ 💬

Reproduce the 1 GB upload with python-dvuploader, then inspect the linked Payara and Apache logs and compare the REST API result with the successful web UI upload. Review related issue #11967 and python-dvuploader issue #47 for context. Done means the API and uploader complete successfully with one file and no duplicate dataset files or 500 error.

Written by the indexing model from the issue text.

Description

When does this issue occur?

What happens?

To whom does it occur (all users, curators, superusers)?

What did you expect to happen?

Contributor guide

First steps

Research direction

Assessment

Problem uploading large files

Description

When does this issue occur?

What happens?

To whom does it occur (all users, curators, superusers)?

What did you expect to happen?

Contributor guide

First steps

Research direction

Assessment

Get new issues in your inbox