ClickHouse / ClickHouse/ClickHouse

Allow virtual hosted s3 bucket regardless of URL pattern

Open
#72,110 7 comments 0 reactions 0 assignees View on GitHub
comp-object-storage external feature
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

**Company or project name**

> An s3 compatible object storage service

**Use case**

> I have an s3 compatible object storage service that does not fit the existing [regex](https://github.com/ClickHouse/ClickHouse/blob/456a41ee4249ace28e36ad65a42c320bb1fdd144/src/IO/S3/URI.cpp#L40) for virtually hosted buckets. The service does not support path style addressing as it is eventually being deprecated.

**Describe the solution you'd like**

> I would like to have a configuration option to influence the addressing style of my bucket. For example, the aws s3 CLI has `addressing_style = virtual`. The requested feature should bypass (or pass) the regex check and use `mybucket.mydomain.com` as the authority. Note that `mydomain.com` may not require a region in a setup such as this.

**Describe alternatives you've considered**

> I can create a new bucket with a name of `mybucket.s3` but would have to migrate data to this new bucket. This also forces me to name my buckets with an `s3` postfix.
> I can match the desired regex and post-process the request before it goes to object storage, but this seems fragile and likely difficult to implement/orchestrate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.