dragonflydb / dragonflydb/dragonfly

Add support for object storage addressing style

Open
#6,045 1 comment 0 reactions 0 assignees View on GitHub
enhancement minor
Dominant language
C++
Stars
31.5k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
137

Description

### Context

With v1.34.1, we're seeing this:

```
I20251014 18:21:49.258402 1 init.cc:127] dragonfly running in opt mode.
* Logs will be written to the first available of the following paths:
/tmp/dragonfly.*
./dragonfly.*
* For the available flags type dragonfly [--help | --helpfull]
* Documentation can be found at: https://www.dragonflydb.io/docs
I20251014 18:21:49.258462 1 dfly_main.cc:757] Starting dragonfly df-v1.34.1-cae4e151e8504b87f3e43956fb31cb337c69ce17
I20251014 18:21:49.258615 1 dfly_main.cc:799] maxmemory has not been specified. Deciding myself....
I20251014 18:21:49.258620 1 dfly_main.cc:808] Found 750.00MiB available memory. Setting maxmemory to 600.00MiB
I20251014 18:21:49.259132 11 uring_proactor.cc:290] IORing with 1024 entries, allocated 98368 bytes, cq_entries is 2048
I20251014 18:21:49.259236 1 proactor_pool.cc:149] Running 1 io threads
I20251014 18:21:49.260867 1 dragonfly_listener.cc:150] SSL version: OpenSSL 3.0.17 1 Jul 2025
I20251014 18:21:49.261096 1 dfly_main.cc:280] Listening on admin socket any:9999
I20251014 18:21:49.261564 1 server_family.cc:1082] Host OS: Linux 6.5.13-65-650-4141-22041-coreweave-amd64-85c45edc x86_64 with 1 threads
I20251014 18:21:49.284922 11 snapshot_storage.cc:510] Creating AWS S3 client; region=us-east-02; https=true; endpoint=cwobject.com
I20251014 18:21:49.284946 11 credentials_provider_chain.cc:28] aws: disabled EC2 metadata
I20251014 18:21:49.285624 11 credentials_provider_chain.cc:36] aws: loaded credentials; provider=environment
I20251014 18:21:49.336686 1 snapshot_storage.cc:567] Load snapshot: Searching for snapshot in S3 path: s3://sa-vectordb-tutorial/dragonfly/
E20251014 18:21:49.343063 1 server_family.cc:1187] Failed to load snapshot with error: Failed list objects in S3 bucket: PathStyleRequestNotAllowed
```
The object storage service we use only allows virtual-host style requests ([AWS S3 CLI config for context](https://docs.aws.amazon.com/cli/latest/topic/s3-config.html#addressing-style)).
Looking at dragonfly code(starting from [here](https://github.com/dragonflydb/dragonfly/blob/a461fbd16677f714398cddb65bfb866539ae1c30/src/server/detail/snapshot_storage.cc#L520) and drilling down), this style should be the default when no style is passed to the `S3ClientConfiguration` constructor. However, requests end up being built using path style. This is confirmed via access logs on the object storage service side.

### The ask

Please add support for specifying addressing style (and corresponding enforcement), to ensure users can configure it and therefore object storage services only supporting a certain style can be used with Dragonfly.

Contributor guide

Open the contributing guide

Research direction

Start in src/server/detail/snapshot_storage.cc around the referenced line, then trace how S3ClientConfiguration is constructed and how requests are built. Compare the current behavior with the AWS S3 addressing-style configuration described in the linked documentation; done means users can select and enforce an addressing style so virtual-host-only object storage services work.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, cpp
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.