influxdata / influxdata/influxdb

Restore from Enteprise backup into OSS should collapse shard groups into a single shard

Open
#22,633 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Proposal:__
By default, shard groups with multiple shards should be collapsed into a single shard when restoring an Enterprise backup into OSS. OSS mostly supports multiple shards per group for queries but does not support creating multiple shards per group and so queries for this case are not well tested.

__Current behavior:__

Reproduction: Create an Enterprise database with RF < # of data nodes. Example:

```
Connected to http://localhost:8086 version 1.9.3-c1.9.3
InfluxDB shell version: unknown
> create database stress
> use stress
Using database stress
> alter retention policy autogen replication factor 1

# write some data in with './inch -host http://localhost:8086 -t 10,10 -p 2000 -time -1000h'

> show shards
...
name: stress
id database retention_policy shard_group start_time end_time expiry_time owners
-- -------- ---------------- ----------- ---------- -------- ----------- ------
5 stress autogen 3 2021-08-23T00:00:00Z 2021-08-30T00:00:00Z 4
6 stress autogen 3 2021-08-23T00:00:00Z 2021-08-30T00:00:00Z 5
7 stress autogen 4 2021-08-30T00:00:00Z 2021-09-06T00:00:00Z 5
8 stress autogen 4 2021-08-30T00:00:00Z 2021-09-06T00:00:00Z 4
9 stress autogen 5 2021-09-06T00:00:00Z 2021-09-13T00:00:00Z 4
10 stress autogen 5 2021-09-06T00:00:00Z 2021-09-13T00:00:00Z 5
11 stress autogen 6 2021-09-13T00:00:00Z 2021-09-20T00:00:00Z 5
12 stress autogen 6 2021-09-13T00:00:00Z 2021-09-20T00:00:00Z 4
13 stress autogen 7 2021-09-20T00:00:00Z 2021-09-27T00:00:00Z 4
14 stress autogen 7 2021-09-20T00:00:00Z 2021-09-27T00:00:00Z 5
15 stress autogen 8 2021-09-27T00:00:00Z 2021-10-04T00:00:00Z 5
16 stress autogen 8 2021-09-27T00:00:00Z 2021-10-04T00:00:00Z 4
17 stress autogen 9 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z 4
18 stress autogen 9 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z 5
```

Take a backup with:

```
/ # mkdir /root/.influxdb/backup1
/ # ./influxd-ctl backup -full /root/.influxdb/backup1
...
```

Start a new OSS instance, and run a restore, e.g.:

```
> ./influxd restore -portable ./backup2/
INFO[0000]log.go:104 gosnowflake.(*defaultLogger).Infof reset OCSP cache file. /Users/sarnold/Library/Caches/Snowflake/ocsp_response_cache.json
INFO[0000]log.go:104 gosnowflake.(*defaultLogger).Infof reading OCSP Response cache file. /Users/sarnold/Library/Caches/Snowflake/ocsp_response_cache.json
2021/10/07 09:53:51 Restoring shard 12 live from backup 20211007T135238Z.s12.tar.gz
2021/10/07 09:53:51 Restoring shard 17 live from backup 20211007T135238Z.s17.tar.gz
2021/10/07 09:53:51 Restoring shard 18 live from backup 20211007T135238Z.s18.tar.gz
2021/10/07 09:53:51 Restoring shard 3 live from backup 20211007T135238Z.s3.tar.gz
2021/10/07 09:53:51 Restoring shard 4 live from backup 20211007T135238Z.s4.tar.gz
2021/10/07 09:53:51 Meta info not found for shard 2 on database _internal. Skipping shard file 20211007T135238Z.s2.tar.gz
2021/10/07 09:53:51 Restoring shard 5 live from backup 20211007T135238Z.s5.tar.gz
2021/10/07 09:53:51 Restoring shard 6 live from backup 20211007T135238Z.s6.tar.gz
2021/10/07 09:53:51 Restoring shard 14 live from backup 20211007T135238Z.s14.tar.gz
2021/10/07 09:53:51 Restoring shard 16 live from backup 20211007T135238Z.s16.tar.gz
2021/10/07 09:53:51 Meta info not found for shard 1 on database _internal. Skipping shard file 20211007T135238Z.s1.tar.gz
2021/10/07 09:53:51 Restoring shard 10 live from backup 20211007T135238Z.s10.tar.gz
2021/10/07 09:53:51 Restoring shard 13 live from backup 20211007T135238Z.s13.tar.gz
2021/10/07 09:53:51 Restoring shard 15 live from backup 20211007T135238Z.s15.tar.gz
2021/10/07 09:53:51 Restoring shard 7 live from backup 20211007T135238Z.s7.tar.gz
2021/10/07 09:53:51 Restoring shard 8 live from backup 20211007T135238Z.s8.tar.gz
2021/10/07 09:53:51 Restoring shard 9 live from backup 20211007T135238Z.s9.tar.gz
2021/10/07 09:53:51 Restoring shard 11 live from backup 20211007T135238Z.s11.tar.gz
```

Show the shards, notice that we have multiple shards per shard group:

```
> show shards
name: _internal
id database retention_policy shard_group start_time end_time expiry_time owners
-- -------- ---------------- ----------- ---------- -------- ----------- ------
1 _internal monitor 1 2021-10-07T00:00:00Z 2021-10-08T00:00:00Z 2021-10-15T00:00:00Z

name: telegraf
id database retention_policy shard_group start_time end_time expiry_time owners
-- -------- ---------------- ----------- ---------- -------- ----------- ------
2 telegraf autogen 2 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z
3 telegraf autogen 2 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z

name: stress
id database retention_policy shard_group start_time end_time expiry_time owners
-- -------- ---------------- ----------- ---------- -------- ----------- ------
4 stress autogen 3 2021-08-23T00:00:00Z 2021-08-30T00:00:00Z
5 stress autogen 3 2021-08-23T00:00:00Z 2021-08-30T00:00:00Z
6 stress autogen 4 2021-08-30T00:00:00Z 2021-09-06T00:00:00Z
7 stress autogen 4 2021-08-30T00:00:00Z 2021-09-06T00:00:00Z
8 stress autogen 5 2021-09-06T00:00:00Z 2021-09-13T00:00:00Z
9 stress autogen 5 2021-09-06T00:00:00Z 2021-09-13T00:00:00Z
10 stress autogen 6 2021-09-13T00:00:00Z 2021-09-20T00:00:00Z
11 stress autogen 6 2021-09-13T00:00:00Z 2021-09-20T00:00:00Z
12 stress autogen 7 2021-09-20T00:00:00Z 2021-09-27T00:00:00Z
13 stress autogen 7 2021-09-20T00:00:00Z 2021-09-27T00:00:00Z
14 stress autogen 8 2021-09-27T00:00:00Z 2021-10-04T00:00:00Z
15 stress autogen 8 2021-09-27T00:00:00Z 2021-10-04T00:00:00Z
16 stress autogen 9 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z
17 stress autogen 9 2021-10-04T00:00:00Z 2021-10-11T00:00:00Z
```

__Desired behavior:__
We should collapse all the shard data for an Enterprise shard group into a single OSS shard during restore by default. Perhaps a flag for the old behaviour could be used.

I believe ImportShard works fine to import new data into an existing shard - if we can validate this then the changes should only be in the OSS restore command, not in the main database.

__Alternatives considered:__
The alternative would be doing a lot of work to validate that all queries/workflows work correctly in OSS with multiple shards per group.

__Use case:__
Enterprise users sometimes want to take their data to OSS 1.x or OSS 2.x, we should enable this with good migration tools.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the OSS restore command and trace how Enterprise shard groups and shard data are imported. Validate whether ImportShard can add data to an existing shard, as the issue suggests. Done means an Enterprise backup restores each shard group into one OSS shard by default, with any old-behavior flag explicitly defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.