apache / apache/arrow-rs-object-store

Allow passing options when using `MultipartStore` API

Open
#301 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

I want to pass options as present in `ObjectStore::put_multipart_opts` to a multipart upload created via the `MultipartStore` interface.

**Describe the solution you'd like**

`MultipartStore` could receive a new method, maybe `create_multipart_opts`, taking something like `PutMultipartOpts` or an entirely new struct.

Annoyingly, it seems like `AmazonS3` takes multipart options at creation, while `MicrosoftAzure` does so at completion...

**Describe alternatives you've considered**

Alternatively, the `put_multipart` API of `ObjectStore` could be used, but I need the ability to resume uploads after restarts, which seems to only be possible with the `MultipartStore` API which provides me with a `MultipartId` I can persist.

Contributor guide

Open the contributing guide

Research direction

Start by reading the MultipartStore interface and ObjectStore::put_multipart_opts, then compare how AmazonS3 and MicrosoftAzure handle multipart options. Define an API that accepts the required options while preserving resumable uploads through a persisted MultipartId; done means the behavior is consistent across the affected implementations.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.