cloudnative-pg / cloudnative-pg/plugin-barman-cloud

[Feature Request] barman-cloud plugin should support virtual-hosted style addressing for S3-compatible storage

Ouverte
#712 2 commentaires 1 réaction 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Go
Étoiles
191
Forks
72
Merge moyen
2 j 21 h
PR mergées (30 j)
21

Description

### Description:
I'm using CloudNativePG with the barman-cloud plugin (version 0.9.0) for PostgreSQL backups to Alibaba Cloud OSS. However, I've encountered a critical issue: the plugin only supports path-style addressing, while Alibaba Cloud OSS (and many other S3-compatible services) now require virtual-hosted style addressing.

### Problem Details:

AWS has deprecated path-style addressing since September 30, 2023
Many cloud providers (Alibaba Cloud OSS, Google Cloud Storage, etc.) require or strongly recommend virtual-hosted style
The barman-cloud plugin doesn't provide any configuration parameter for addressing_style
When endpointURL is specified, it defaults to path-style addressing

### Current Workaround:
Currently, users have to:

Use custom backup solutions instead of the official plugin
Or use unsupported path-style endpoints (if available)

### Expected Behavior:
The barman-cloud plugin should support both addressing styles with a configuration parameter like:

```
configuration:
destinationPath: "s3://bucket-name/"
endpointURL: "https://bucket-name.oss-cn-region.aliyuncs.com"
s3AddressingStyle: "virtual" # or "path"
```

Or automatically detect the appropriate style based on the endpoint URL format.

### Technical Background:

- AWS SDK (boto3) already supports addressing_style parameter in boto3.session.Config

- Virtual-hosted style is now the standard for S3-compatible services

- Path-style addressing has been deprecated by AWS

### Impact:
This limitation prevents users from using the official plugin with:

- Alibaba Cloud OSS

- Google Cloud Storage

- DigitalOcean Spaces

- And other S3-compatible services that enforce virtual-hosted style

### Proposed Solution:
Add a new parameter s3AddressingStyle to the ObjectStore configuration that gets passed to the underlying boto3 client configuration.

### Code Reference:
In barman-cloud implementation, it should configure boto3 like:

```
config = boto3.session.Config(
s3={'addressing_style': addressing_style}
)
s3_client = boto3.client(
's3',
endpoint_url=endpoint,
config=config
)
```

### Environment:

- CloudNativePG: 1.28.0

- barman-cloud plugin: 0.9.0

- Kubernetes: 1.32.7

- Storage: Alibaba Cloud OSS

### Additional Context:

1. Attempted Solutions:

[ ] I have tried setting the environment variable AWS_S3_ADDRESSING_STYLE=virtual in the pod configuration, but it has no effect

- Tested various endpointURL formats including:

- https://bucket-name.oss-cn-region.aliyuncs.com (virtual-hosted style)

- https://oss-cn-region.aliyuncs.com (path-style)

- The plugin consistently uses path-style addressing regardless of the endpoint URL format

2. Current Status:

- Alibaba Cloud OSS strictly enforces virtual-hosted style addressing

- AWS has completely phased out path-style support

- This issue is becoming critical as more S3-compatible services adopt virtual-hosted style

3. Reproduction Steps:

- Configure barman-cloud plugin with Alibaba Cloud OSS endpoint

- Set endpointURL to virtual-hosted style format

- Observe that the plugin still attempts path-style requests

- Receive errors from OSS indicating path-style is not supported

4. Workaround Limitations:

- Custom backup scripts lack the integration and reliability of the official plugin

- Alternative S3-compatible endpoints may not be available in all regions

- This forces users to choose between functionality and using the official tooling

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par localiser la configuration barman-cloud ObjectStore et la construction du client boto3 décrites dans l'issue. Suivez la manière dont endpointURL est transmis au client, puis vérifiez qu'un style d'adressage configurable prend en charge à la fois les requêtes virtual-hosted et path-style pour un stockage compatible avec S3.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
aws, go, postgresql, python
Domaine
cloud, database
Type d'issue
Fonctionnalité
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
55/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.