cloudnative-pg / cloudnative-pg/plugin-barman-cloud

[Feature Request] barman-cloud plugin should support virtual-hosted style addressing for S3-compatible storage

Open
#712 2 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
191
Forks
72
Avg merge
2d 21h
Merged PRs (30d)
21

Description

### Description:
I'm using CloudNativePG with the barman-cloud plugin (version 0.9.0) for PostgreSQL backups to Alibaba Cloud OSS. However, I've encountered a critical issue: the plugin only supports path-style addressing, while Alibaba Cloud OSS (and many other S3-compatible services) now require virtual-hosted style addressing.

### Problem Details:

AWS has deprecated path-style addressing since September 30, 2023
Many cloud providers (Alibaba Cloud OSS, Google Cloud Storage, etc.) require or strongly recommend virtual-hosted style
The barman-cloud plugin doesn't provide any configuration parameter for addressing_style
When endpointURL is specified, it defaults to path-style addressing

### Current Workaround:
Currently, users have to:

Use custom backup solutions instead of the official plugin
Or use unsupported path-style endpoints (if available)

### Expected Behavior:
The barman-cloud plugin should support both addressing styles with a configuration parameter like:

```
configuration:
destinationPath: "s3://bucket-name/"
endpointURL: "https://bucket-name.oss-cn-region.aliyuncs.com"
s3AddressingStyle: "virtual" # or "path"
```

Or automatically detect the appropriate style based on the endpoint URL format.

### Technical Background:

- AWS SDK (boto3) already supports addressing_style parameter in boto3.session.Config

- Virtual-hosted style is now the standard for S3-compatible services

- Path-style addressing has been deprecated by AWS

### Impact:
This limitation prevents users from using the official plugin with:

- Alibaba Cloud OSS

- Google Cloud Storage

- DigitalOcean Spaces

- And other S3-compatible services that enforce virtual-hosted style

### Proposed Solution:
Add a new parameter s3AddressingStyle to the ObjectStore configuration that gets passed to the underlying boto3 client configuration.

### Code Reference:
In barman-cloud implementation, it should configure boto3 like:

```
config = boto3.session.Config(
s3={'addressing_style': addressing_style}
)
s3_client = boto3.client(
's3',
endpoint_url=endpoint,
config=config
)
```

### Environment:

- CloudNativePG: 1.28.0

- barman-cloud plugin: 0.9.0

- Kubernetes: 1.32.7

- Storage: Alibaba Cloud OSS

### Additional Context:

1. Attempted Solutions:

[ ] I have tried setting the environment variable AWS_S3_ADDRESSING_STYLE=virtual in the pod configuration, but it has no effect

- Tested various endpointURL formats including:

- https://bucket-name.oss-cn-region.aliyuncs.com (virtual-hosted style)

- https://oss-cn-region.aliyuncs.com (path-style)

- The plugin consistently uses path-style addressing regardless of the endpoint URL format

2. Current Status:

- Alibaba Cloud OSS strictly enforces virtual-hosted style addressing

- AWS has completely phased out path-style support

- This issue is becoming critical as more S3-compatible services adopt virtual-hosted style

3. Reproduction Steps:

- Configure barman-cloud plugin with Alibaba Cloud OSS endpoint

- Set endpointURL to virtual-hosted style format

- Observe that the plugin still attempts path-style requests

- Receive errors from OSS indicating path-style is not supported

4. Workaround Limitations:

- Custom backup scripts lack the integration and reliability of the official plugin

- Alternative S3-compatible endpoints may not be available in all regions

- This forces users to choose between functionality and using the official tooling

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.