jupyter / jupyter/notebook

Uploading a large file through contents API

Open
#5,705 1 comment 4 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
13.3k
Forks
5.8k
Avg merge
6d 11h
Merged PRs (30d)
7

Description

Hey! I'm trying to upload a large file (let's say 5GB) through contents API.

The problem I'm facing is that as the API endpoint only accepts a JSON payload with a `"content": string` key, it is pretty difficult to upload a file this big.

The problem is that for creating the JSON payload, I would have to read the file contents into memory first. I've tried also using a stream as the payload and streaming the data into the request, but Jupyter quickly returns an `HTTP 413 Payload Too Large`.

What would be your recommended way of getting a file this big inside the notebook workspace?

The way I'd expect the file upload endpoint to work is that it would accept an `application/x-www-form-urlencoded` payload with the file data inside of it. Allowing data in this format would also enable the API endpoint to be used with a standard HTML form.

Happy to help with a PR if you find this way of uploading large files worth it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.