cloudnative-pg / cloudnative-pg/plugin-barman-cloud
MEGA S4 + Inconsistent part size
- Ngôn ngữ chính
- Go
- Star
- 191
- Fork
- 72
- Merge trung bình
- 1 ngày 16 giờ
- Pull request đã merge (30 ngày)
- 18
Mô tả
Hi,
I've noticed that my backups haven't been working for a while now and I just can't get it to work anymore. I use MEGA S4 as an S3 object storage provider with this configuration
```yaml
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: my-objectstore-name
namespace: mynamespace
spec:
configuration:
data:
compression: snappy
jobs: 1
destinationPath: s3://my-bucket-name/
endpointURL: https://s3.eu-luxembourg.megas4.com
s3Credentials:
accessKeyId:
key: ACCESS_KEY_ID
name: mega-s3
secretAccessKey:
key: ACCESS_SECRET_KEY
name: mega-s3
wal:
compression: snappy
maxParallel: 1
logLevel: info
retentionPolicyIntervalSeconds: 1800
retentionPolicy: 14d
```
WAL uploads are still working fine. But on-demand backups and scheduled backups are failing with this error message
```
failure uploading data (An error occurred (InvalidPart) when calling the UploadPart operation: Inconsistent part size. S4 requires parts to have the same size (except a smaller size can be allowed for the last part) and consecutive sequential part numbers starting by 1)
```
The full json output
```json
{"level":"info","ts":"2026-06-24T16:48:07.752576576Z","logger":"barman-cloud-backup","msg":"2026-06-24 16:48:07,752 [104] ERROR: Backup failed uploading data (An error occurred (InvalidPart) when calling the UploadPart operation: Inconsistent part size. S4 requires parts to have the same size (except a smaller size can be allowed for the last part) and consecutive sequential part numbers starting by 1)","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.893659577Z","logger":"barman-cloud-backup","msg":"Exception ignored in: ","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.893914989Z","logger":"barman-cloud-backup","msg":"Traceback (most recent call last):","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.894077144Z","logger":"barman-cloud-backup","msg":" File \"/usr/lib/python3.13/tarfile.py\", line 415, in __del__","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897247336Z","logger":"barman-cloud-backup","msg":" self.close()","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897379955Z","logger":"barman-cloud-backup","msg":" File \"/usr/lib/python3.13/tarfile.py\", line 466, in close","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897399926Z","logger":"barman-cloud-backup","msg":" self.fileobj.write(self.buf)","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897432384Z","logger":"barman-cloud-backup","msg":" File \"/venv/lib/python3.13/site-packages/barman/cloud.py\", line 247, in write","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897481828Z","logger":"barman-cloud-backup","msg":" if self.buffer and self.buffer.tell() > self.chunk_size:","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897539826Z","logger":"barman-cloud-backup","msg":" File \"/usr/lib/python3.13/tempfile.py\", line 690, in func_wrapper","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897543751Z","logger":"barman-cloud-backup","msg":" return func(*args, **kwargs)","pipe":"stderr","logging_pod":"my-db-1"}
{"level":"info","ts":"2026-06-24T16:48:08.897551773Z","logger":"barman-cloud-backup","msg":"ValueError: I/O operation on closed file","pipe":"stderr","logging_pod":"my-db-1"}
```
I'm pretty sure it has worked in the past. I've been searching a lot, but I haven't been able to find anything that might help me. Does anyone have any ideas? I think this might be something specific to MEGA. Does anyone know of a workaround for this, or has anyone seen something like this before?
Thanks in advance for any advice!
The version of the plugin being used is: **v0.13.0**
Best regards,
Paul
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu bằng cách kiểm tra đường dẫn ghi trong barman/cloud.py được nêu trong traceback và luồng tải lên bản sao lưu được plugin v0.13.0 sử dụng. Tái hiện lỗi với cấu hình ObjectStore được cung cấp và MEGA S4, sau đó so sánh hành vi multipart với kích thước part mà nhà cung cấp yêu cầu. Công việc được hoàn tất khi các bản sao lưu theo yêu cầu và theo lịch đều được tải lên thành công mà không làm hỏng đường dẫn tải lên WAL.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- python
- Lĩnh vực
- cloud, databases
- Loại issue
- Lỗi
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Cần làm rõ
- Mức phù hợp với người mới
- 35/100