Many problems about fdbrestore --remove_prefix
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
I tried `fdbrestore --remove_prefix` with foundation 6.0.15, there are many
problems with it. I think these problem still exist.
### 1. unknown option `--remove_prefix'
the help info of fdbrestore (`fdbrestore --help`) says:
```
--remove_prefix PREFIX prefix to remove from the restored keys
```
but it's in fact `-remove_prefix`, see source file `fdbbackup/backup.actor.cpp`.
also same problem for `--add_prefix`
### 2. the prefix of `-remove_prefix` and `-add_prefix` doesn't do escaping, but key in argument `--keys` does
### 3. `-remove_prefix` could not work without argument `--keys`
If there is no `--keys` when calling `fdbrestore`, it will add a default range named `normalKeys`. The `normalKeys` is ["", \xFF) .
Then it will fail the assert in `removePrefix()` at `Arena.h`, because the size of "" is less than size of prefix.
### 4. the prefix must bigger then begin key of the restore range
For example, restore key range is [xxx1, xxx2) and remove_prefix is "xxx". This will fail the assert of `restoreRange.contains(removePrefix)` in `submitRestore()` at `FileBackupAgent.actor.cpp`.
### 5. must be careful about the end key of the restore range
For example, restore key range is [aaaaa, zzzzz), add_prefix is "xxxxx" and remove_prefix is "ccccc". And the database contains two keys: "ccccc1", "ccccc2". This will fail with error "'Range begin key larger than end key' on 'restore_range_data'".
The reason is the logic to calculate the `fileEnd` at `FileBackupAgent.actor.cpp` is buggy.
The `fileRange` may be [xxxxx1, xxxxx), then trigger the error before.
To make it work, user should set the restore key range to [aaaaa, zzzzz\xff).
Contributor guide
Research direction
Start in fdbbackup/backup.actor.cpp and FileBackupAgent.actor.cpp, then inspect removePrefix() in Arena.h. Reproduce the documented fdbrestore cases with --remove_prefix, --add_prefix, and --keys, including the range examples. Done means the option names, prefix escaping, missing --keys behavior, range validation, and fileEnd calculation no longer produce the reported failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100