apple / apple/foundationdb

Many problems about fdbrestore --remove_prefix

Open
#1,537 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

I tried `fdbrestore --remove_prefix` with foundation 6.0.15, there are many
problems with it. I think these problem still exist.

### 1. unknown option `--remove_prefix'

the help info of fdbrestore (`fdbrestore --help`) says:

```
--remove_prefix PREFIX prefix to remove from the restored keys
```

but it's in fact `-remove_prefix`, see source file `fdbbackup/backup.actor.cpp`.

also same problem for `--add_prefix`

### 2. the prefix of `-remove_prefix` and `-add_prefix` doesn't do escaping, but key in argument `--keys` does

### 3. `-remove_prefix` could not work without argument `--keys`

If there is no `--keys` when calling `fdbrestore`, it will add a default range named `normalKeys`. The `normalKeys` is ["", \xFF) .

Then it will fail the assert in `removePrefix()` at `Arena.h`, because the size of "" is less than size of prefix.

### 4. the prefix must bigger then begin key of the restore range

For example, restore key range is [xxx1, xxx2) and remove_prefix is "xxx". This will fail the assert of `restoreRange.contains(removePrefix)` in `submitRestore()` at `FileBackupAgent.actor.cpp`.

### 5. must be careful about the end key of the restore range

For example, restore key range is [aaaaa, zzzzz), add_prefix is "xxxxx" and remove_prefix is "ccccc". And the database contains two keys: "ccccc1", "ccccc2". This will fail with error "'Range begin key larger than end key' on 'restore_range_data'".

The reason is the logic to calculate the `fileEnd` at `FileBackupAgent.actor.cpp` is buggy.
The `fileRange` may be [xxxxx1, xxxxx), then trigger the error before.

To make it work, user should set the restore key range to [aaaaa, zzzzz\xff).

Contributor guide

Open the contributing guide

Research direction

Start in fdbbackup/backup.actor.cpp and FileBackupAgent.actor.cpp, then inspect removePrefix() in Arena.h. Reproduce the documented fdbrestore cases with --remove_prefix, --add_prefix, and --keys, including the range examples. Done means the option names, prefix escaping, missing --keys behavior, range validation, and fileEnd calculation no longer produce the reported failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.