Support using multipart to upload files to S3
Open
@Lloyd-Pottiger is already working on this.
Since Nov 10, 2023.
type/feature-request
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Feature Request
Is your feature request related to a problem? Please describe:
Now, we need to create a file locally and then upload it to S3 entirely.
Describe the feature you'd like:
Using S3 multipart SDK to upload a single object as a set of parts so that we do not need to write data into the local disk.
Describe alternatives you've considered:
With this feature, TiFlash can get some improvement in the following aspects:
- Performance
- Do not need to write local disk
- Can concurrently upload different parts
- Stability
- If one part upload failed, just retransmit that part without affecting other parts
Teachability, Documentation, Adoption, Migration Strategy:
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.