cloudnative-pg / cloudnative-pg/plugin-barman-cloud

Base backups are single-threaded and ignore concurrency settings

Open
#906 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
191
Forks
72
Avg merge
1d 16h
Merged PRs (30d)
18

Description

The `plugin-barman-cloud` performs base backups using a single upload stream. Unlike WAL archiving (which supports `maxParallel`), there is no way to parallelize the data backup phase. This results in bottlenecks for large databases.

This option:
```
$ kubectl explain objectstore.spec.configuration.data.jobs
GROUP: barmancloud.cnpg.io
KIND: ObjectStore
VERSION: v1

FIELD: jobs

DESCRIPTION:
The number of parallel jobs to be used to upload the backup, defaults
to 2
```
is not applicable due to limitation (only for rsync):
https://github.com/EnterpriseDB/barman/blob/ff040cd1feee805da3e4662a187c583f8730dfab/docs/user_guide/configuration.rst?plain=1#L336

> parallel_jobs

> Controls the number of parallel workers used to copy files during backup or recovery.
It must be a positive integer. Default is ``1``.

> note::
> Applies only when ``backup_method = rsync``.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the plugin-barman-cloud base-backup upload path and compare it with WAL archiving's maxParallel handling. Determine how the data upload can honor a configurable parallel-job setting; done means the base backup phase supports parallel uploads without changing the existing default behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
cloud, databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.