dropbox / dropbox/dropbox-sdk-python

Allow chunked uploads (and maybe even streaming uploads)

未关闭
#459 4 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
983
派生
330
平均合并
3 分钟
30 天内合并 PR
10

描述

**Why is this feature valuable to you? Does it solve a problem you're having?**
The requests library allows both streaming and chunked uploads (see https://requests.readthedocs.io/en/latest/user/advanced/#streaming-uploads and https://requests.readthedocs.io/en/latest/user/advanced/#chunk-encoded-requests). This has two benefits:

1. It is sufficient to only load small parts of a file into memory before upload.
2. It is possible to limit bandwidth usage by using a generator that provides chunks at a limited rate.

The Dropbox API of course already requires upload sessions (`files/upload_session_start`, `files/upload_session_append` and `files/upload_session_finish`) to upload files > 150 MB. However, this approach by itself does not replace chunked or streaming uploads because:

1. The request body should be ideally >= 4 MB to reduce the total number of API calls (both for efficiency and to not exhaust data transport API call limits).
2. Bandwidth control will be very coarse when performed on chunks of 4 MB compared for example 2 kB.
4. Memory usage will still be larger compared to 1 kB or 2 kB chunks, especially for parallel uploads.

**Describe the solution you'd like**
Requests supports streaming uploads by passing a file-like object as the request body and chunked uploads by passing a generator as the request body. However, the Python SDK explicitly prevents both by requiring the request body to be of type `bytes`:

https://github.com/dropbox/dropbox-sdk-python/blob/9895d705317583cedb9fc11e5aa1f17f6bea303a/dropbox/dropbox_client.py#L533-L539

It would be good to either completely drop this limitation, with appropriate warnings in the doc string, or at least allow chunked uploads (where requests handles retry / rewind logic) even when disallowing streaming uploads.

**Describe alternatives you've considered**
Not at present.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。