cloudnative-pg / cloudnative-pg/plugin-barman-cloud

Failure to restore from backup due to missing WAL file

オープン
#618 コメント 1 件 リアクション 2 件 担当者 0 名 GitHub で見る
bug
主要言語
Go
スター
191
フォーク
72
平均マージ
2日 21時間
マージ済み PR(30日)
21

説明

I created a backup, when it was checked as finished I tried deleting and restoring the cluster but it got stuck in a state trying to find a missing WAL file.

I couldn't find a way to load only from the base and ignore the WAL files completely either.

This is how my files were set up

Only WAL present in AWS was `/wals/0000000600000000/0000000600000000000000AA.gz`

backup.info had
```
backup_label='START WAL LOCATION: 0/AB000028 (file 0000000600000000000000AB)\nCHECKPOINT LOCATION: 0/AB000028\nBACKUP METHOD: streamed\nBACKUP FROM: standby\nSTART TIME: 2025-10-24 02:26:08 UTC\nLABEL: Barman backup cloud 20251024T022608\nSTART TIMELINE: 6\n'
backup_name=backup-20251024022604
begin_offset=40
begin_time=2025-10-24 02:26:08.217166+00:00
begin_wal=0000000600000000000000AB
begin_xlog=0/AB000028
children_backup_ids=None
cluster_size=48071936
compression=None
config_file=/var/lib/postgresql/data/pgdata/postgresql.conf
copy_stats={'total_time': 3.17413, 'number_of_workers': 2, 'analysis_time': 0, 'analysis_time_per_item': {'data': 0}, 'copy_time_per_item': {'data': 1.796744}, 'serialized_copy_time_per_item': {'data': 1.924254}, 'copy_time': 1.796744, 'serialized_copy_time': 1.924254}
data_checksums=off
deduplicated_size=None
encryption=None
end_offset=416
end_time=2025-10-24 02:26:11.034605+00:00
end_wal=0000000600000000000000AB
end_xlog=0/AB0001A0
error=None
hba_file=/var/lib/postgresql/data/pgdata/pg_hba.conf
ident_file=/var/lib/postgresql/data/pgdata/pg_ident.conf
included_files=['/var/lib/postgresql/data/pgdata/custom.conf', '/var/lib/postgresql/data/pgdata/override.conf']
mode=None
parent_backup_id=None
pgdata=/var/lib/postgresql/data/pgdata
server_name=cloud
size=None
status=DONE
summarize_wal=off
systemid=7564138978558590997
tablespaces=None
timeline=6
version=170006
xlog_segment_size=16777216
```

inside `data.tar` the `backup_label` had:

```
START WAL LOCATION: 0/AB000028 (file 0000000600000000000000AB)
CHECKPOINT LOCATION: 0/AB000028
BACKUP METHOD: streamed
BACKUP FROM: standby
START TIME: 2025-10-24 02:26:08 UTC
LABEL: Barman backup cloud 20251024T022608
START TIMELINE: 6
```

Pod log
```
{"level":"info","ts":"2025-10-24T02:46:04.754995341Z","msg":"Starting webserver","logging_pod":"postgres-cluster-1-full-recovery","address":"localhost:8010","hasTLS":false}
{"level":"info","ts":"2025-10-24T02:46:04.860377215Z","msg":"Restore through plugin detected, proceeding...","logging_pod":"postgres-cluster-1-full-recovery"}
{"level":"error","ts":"2025-10-24T02:46:12.557539207Z","msg":"Error while restoring a backup","logging_pod":"postgres-cluster-1-full-recovery","error":"rpc error: code = Unknown desc = encountered an error while checking the presence of first needed WAL in the archive: object storage or file not found 0000000600000000000000AB: WAL not found","stacktrace":"github.com/cloudnative-pg/machinery/pkg/log.(*logger).Error\n\tpkg/mod/github.com/cloudnative-pg/machinery@v0.3.1/pkg/log/log.go:125\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/instance/restore.restoreSubCommand\n\tinternal/cmd/manager/instance/restore/restore.go:79\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/instance/restore.(*restoreRunnable).Start\n\tinternal/cmd/manager/instance/restore/restore.go:62\nsigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.22.3/pkg/manager/runnable_group.go:260"}
{"level":"info","ts":"2025-10-24T02:46:12.569395329Z","msg":"Stopping and waiting for non leader election runnables"}
{"level":"info","ts":"2025-10-24T02:46:12.569421782Z","msg":"Stopping and waiting for leader election runnables"}
{"level":"info","ts":"2025-10-24T02:46:12.574760522Z","msg":"Webserver exited","logging_pod":"postgres-cluster-1-full-recovery","address":"localhost:8010"}
{"level":"info","ts":"2025-10-24T02:46:12.57576866Z","msg":"Stopping and waiting for caches"}
{"level":"info","ts":"2025-10-24T02:46:12.574699681Z","msg":"Stopping and waiting for warmup runnables"}
{"level":"info","ts":"2025-10-24T02:46:12.578650618Z","msg":"Stopping and waiting for webhooks"}
{"level":"info","ts":"2025-10-24T02:46:12.578679668Z","msg":"Stopping and waiting for HTTP servers"}
{"level":"info","ts":"2025-10-24T02:46:12.578689889Z","msg":"Wait completed, proceeding to shutdown the manager"}
{"level":"error","ts":"2025-10-24T02:46:12.578836461Z","msg":"restore error","logging_pod":"postgres-cluster-1-full-recovery","error":"while restoring cluster: rpc error: code = Unknown desc = encountered an error while checking the presence of first needed WAL in the archive: object storage or file not found 0000000600000000000000AB: WAL not found","stacktrace":"github.com/cloudnative-pg/machinery/pkg/log.(*logger).Error\n\tpkg/mod/github.com/cloudnative-pg/machinery@v0.3.1/pkg/log/log.go:125\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/instance/restore.NewCmd.func1\n\tinternal/cmd/manager/instance/restore/cmd.go:101\ngithub.com/spf13/cobra.(*Command).execute\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1015\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1148\ngithub.com/spf13/cobra.(*Command).Execute\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1071\nmain.main\n\tcmd/manager/main.go:71\nruntime.main\n\t/opt/hostedtoolcache/go/1.25.3/x64/src/runtime/proc.go:285"}
```

My ObjectStore

```
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: postgres-backup-s3
namespace: superset
spec:
configuration:
destinationPath: s3://withub-backup-cloudnative-postgres/superset
s3Credentials:
inheritFromIAMRole: true
wal:
compression: gzip
retentionPolicy: 30d
status:
serverRecoveryWindow:
postgres-cluster-restored:
firstRecoverabilityPoint: '2025-10-24T02:18:01Z'
lastSuccessfulBackupTime: '2025-10-24T02:26:11Z'
```

My Backup CRD

```
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
labels:
cnpg.io/cluster: postgres-cluster
cnpg.io/immediateBackup: 'true'
cnpg.io/scheduled-backup: postgres-scheduled-backup
name: postgres-scheduled-backup-20251024021602
namespace: superset
spec:
cluster:
name: postgres-cluster
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
status:
backupId: 20251024T022608
backupName: backup-20251024022604
beginLSN: 0/AB000028
beginWal: 0000000600000000000000AB
endLSN: 0/AB0001A0
endWal: 0000000600000000000000AB
instanceID:
ContainerID: >-
containerd://2594945321f8c0aca511cf07ee38eea4e303253888ae84e1219f2b80dab5ef5c
podName: postgres-cluster-1
majorVersion: 17
method: plugin
online: true
phase: completed
pluginMetadata:
clusterUID: e6e34f8d-e925-437f-8e98-dd6425e10a03
displayName: BarmanCloudInstance
name: barman-cloud.cloudnative-pg.io
pluginName: barman-cloud.cloudnative-pg.io
timeline: '6'
version: 0.7.0
startedAt: '2025-10-24T02:26:08Z'
stoppedAt: '2025-10-24T02:26:11Z'
```

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start with internal/cmd/manager/instance/restore/restore.go at the WAL-presence check around line 79, then trace the plugin restore path using the supplied logs, backup metadata, and ObjectStore configuration. Investigate why the first required WAL is considered absent despite the completed backup, and verify the result with a restore reproduction or regression coverage that establishes the expected handling.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, go, postgresql
領域
cloud, databases
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。