plotly / plotly/dash

dcc.Upload errors with big files

Open
#2,869 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug feature P3
Dominant language
Python
Stars
24.4k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
13

Description

Context

Issues

  • If I run the app with gunicorn or in my local env, some time after I upload the file the tab freezes or I get an internal server error (NOT a timeout because dcc.Upload has some kind of async functionality). > This is similar to what this issue describes: https://github.com/plotly/dash/issues/1728
Screenshot 2024-05-28 at 10 13 32 - If I run the app with Python in a Dash Enterprise workspace, I get a callback error which is kind of weird. If you look at the callback code and the print statements in the terminal, we only get two values despite having three Inputs. The first one is probably an empty string or list, which causes the if statement to “fail” (since list_of_contents is not None), so the error we see is related to code after the if statement. The real problem is that, even if the filenames and dates get passed correctly, the content is empty > This issue has been described before in the forum: (https://community.plotly.com/t/upload-500mb-csv-but-content-is-empty/83297. Screenshot error Screenshot callback code

Additional context

  • I've tried dash-uploader (link) and it works okay but I’m concerned about (1) privacy (all files are accessible by all users by default, as opposed to session-specific storage) and (2) memory (how does saving many big files affect the app memory/performance)
  • dash-uploader is not an official component and it’s not maintained, which raises some concerns regarding production environment.

Request

  • dcc.Upload being able to handle big files (500MB, >1GB, etc).
  • If the file is not been processed correctly (list_of_contents is empty) show a more helpful error.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the dcc.Upload documentation sample and reproduce the behavior using files over 500MB in the local, gunicorn, and Dash Enterprise environments described. Compare the callback inputs when content is empty or missing. Done means large uploads are handled reliably and an unsuccessful upload produces a helpful error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.