Aiven-Open / Aiven-Open/pghoard

Unable to restore from backup

Open
#214 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
111
Avg merge
1d 5h
Merged PRs (30d)
2

Description

I've tried multiple times to accomplish a basic scenario.

1. Start `pghoard`, which initially takes the first base backup
2. Restore that backup to new instance

I'm using S3 storage to save the backups.

Every time I do restore I get errors about miising files in the backup storage

```
2017-07-04 09:17:07,531 TransferAgent Thread-9 WARNING 'default/timeline/0000001C.history' not found from storage
2017-07-04 09:17:07,531 TransferAgent Thread-9 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001C.history', size: 0, took 0.028s
2017-07-04 09:17:07,531 TransferAgent Thread-8 WARNING 'default/timeline/0000001D.history' not found from storage
2017-07-04 09:17:07,531 TransferAgent Thread-8 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001D.history', size: 0, took 0.028s
2017-07-04 09:17:07,535 TransferAgent Thread-12 WARNING 'default/timeline/0000001E.history' not found from storage
2017-07-04 09:17:07,535 TransferAgent Thread-12 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001E.history', size: 0, took 0.030s
2017-07-04 09:17:07,535 TransferAgent Thread-10 WARNING 'default/timeline/0000001A.history' not found from storage
2017-07-04 09:17:07,535 TransferAgent Thread-10 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001A.history', size: 0, took 0.030s
2017-07-04 09:17:07,599 TransferAgent Thread-11 WARNING 'default/timeline/0000001B.history' not found from storage
2017-07-04 09:17:07,600 TransferAgent Thread-11 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001B.history', size: 0, took 0.098s
127.0.0.1 - - [04/Jul/2017 09:17:07] "GET /default/archive/0000001A.history HTTP/1.1" 404 -
2017-07-04 09:17:07,677 TransferAgent Thread-8 WARNING 'default/timeline/00000019.history' not found from storage
2017-07-04 09:17:07,677 TransferAgent Thread-8 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/00000019.history', size: 0, took 0.008s
2017-07-04 09:17:07,678 TransferAgent Thread-9 WARNING 'default/timeline/0000001A.history' not found from storage
2017-07-04 09:17:07,678 TransferAgent Thread-9 INFO 'DOWNLOAD' FAILED transfer of key: 'default/timeline/0000001A.history', size: 0, took 0.009s
127.0.0.1 - - [04/Jul/2017 09:17:07] "GET /default/archive/00000019.history HTTP/1.1" 404 -
2017-07-04 09:17:07,758 TransferAgent Thread-12 WARNING 'default/xlog/0000001900000000000000F0' not found from storage
2017-07-04 09:17:07,758 TransferAgent Thread-12 INFO 'DOWNLOAD' FAILED transfer of key: 'default/xlog/0000001900000000000000F0', size: 0, took 0.012s
2017-07-04 09:17:07,760 TransferAgent Thread-9 WARNING 'default/xlog/0000001900000000000000EF' not found from storage
2017-07-04 09:17:07,760 TransferAgent Thread-9 INFO 'DOWNLOAD' FAILED transfer of key: 'default/xlog/0000001900000000000000EF', size: 0, took 0.012s
2017-07-04 09:17:07,801 TransferAgent Thread-11 WARNING 'default/xlog/0000001900000000000000F2' not found from storage
2017-07-04 09:17:07,802 TransferAgent Thread-11 INFO 'DOWNLOAD' FAILED transfer of key: 'default/xlog/0000001900000000000000F2', size: 0, took 0.055s
2017-07-04 09:17:07,810 TransferAgent Thread-8 INFO 'DOWNLOAD' transfer of key: 'default/xlog/0000001900000000000000F3', size: 797270, took 0.062s
2017-07-04 09:17:07,817 TransferAgent Thread-10 WARNING 'default/xlog/0000001900000000000000F1' not found from storage
2017-07-04 09:17:07,818 TransferAgent Thread-10 INFO 'DOWNLOAD' FAILED transfer of key: 'default/xlog/0000001900000000000000F1', size: 0, took 0.071s
127.0.0.1 - - [04/Jul/2017 09:17:07] "GET /default/archive/0000001900000000000000EF HTTP/1.1" 404 -
```

The thing is that those history/xlog files don't exist in the backup, and `pghoard` never uploaded them. I just see the base backup archive in the S3. What could be the reason.

I tried multiple combination of restore, including `--restore-to-master`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.