apache / apache/arrow-rs-object-store

object-store: support for client-side encryption on S3

Open
#294 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

On S3 when objects are [client-side encrypted](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingClientSideEncryption.html) the [encryption metadata is stored and transmitted through headers](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/package-summary.html#:~:text=S3%201st%20gen%20crypto%20meta,time%20randomly%20generated%20data%20key) on GET and HEAD requests, which is required to decrypt objects on the client-side.

Currently the `ObjectMeta` does not support provider-specific metadata although it's already able to extract it from request headers.

**Describe the solution you'd like**
The minimal change would be to passthrough S3-specific client-side encryption metadata through the `ObjectMeta`, which could then by used by client.

**Describe alternatives you've considered**
The better solution would be to support content decryption within the library itself, however it would create dependencies on `openssl` or similar and add considerable amount of logic, which might limit the types of clients who would be able to use this library.

**Additional context**
I'm willing to author the change, but need guidance on the design.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how S3 GET and HEAD request headers are extracted into ObjectMeta. Review the existing provider-specific metadata handling and the linked AWS client-side encryption header documentation. Done means the relevant S3 encryption metadata is passed through ObjectMeta for clients without adding decryption logic.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.