cloudnative-pg / cloudnative-pg/plugin-barman-cloud
Base backups are single-threaded and ignore concurrency settings
- Dominant language
- Go
- Stars
- 191
- Forks
- 72
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 18
Description
The `plugin-barman-cloud` performs base backups using a single upload stream. Unlike WAL archiving (which supports `maxParallel`), there is no way to parallelize the data backup phase. This results in bottlenecks for large databases.
This option:
```
$ kubectl explain objectstore.spec.configuration.data.jobs
GROUP: barmancloud.cnpg.io
KIND: ObjectStore
VERSION: v1
FIELD: jobs
DESCRIPTION:
The number of parallel jobs to be used to upload the backup, defaults
to 2
```
is not applicable due to limitation (only for rsync):
https://github.com/EnterpriseDB/barman/blob/ff040cd1feee805da3e4662a187c583f8730dfab/docs/user_guide/configuration.rst?plain=1#L336
> parallel_jobs
> Controls the number of parallel workers used to copy files during backup or recovery.
It must be a positive integer. Default is ``1``.
> note::
> Applies only when ``backup_method = rsync``.
Contributor guide
Research direction
Start by tracing the plugin-barman-cloud base-backup upload path and compare it with WAL archiving's maxParallel handling. Determine how the data upload can honor a configurable parallel-job setting; done means the base backup phase supports parallel uploads without changing the existing default behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, postgresql
- Domain
- cloud, databases, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100