pingcap / pingcap/tidb-operator

Support other than gzip in tidb-lightning

Open
#2,815 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status/help-wanted
Dominant language
Go
Stars
1.3k
Forks
540
Avg merge
3d 2h
Merged PRs (30d)
18

Description

Feature Request

Is your feature request related to a problem? Please describe:

The Init Container of tidb-lightning is pingcap/tidb-cloud-backup and it seems to have tar xvzf hardcoded into its Command. That means that it'll fail if a different compression scheme is used.

I guess this means the pingcap/tidb-cloud-backup image needs to be changed, but I have no idea where that image comes from.

Init Containers:
  data-retriever:
    Container ID:  docker://42b42f516a3bf5cf66c4c414936a5dd8c7f1266639c7cd0a19fa398782fc75b5
    Image:         pingcap/tidb-cloud-backup:20200229
    Image ID:      docker-pullable://pingcap/tidb-cloud-backup@sha256:d4c61b3573882c3d00e5fee3f911123ed84bdc7235ba8acbbab56560f2433cf1
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
      set -euo pipefail
      filename=$(basename s3://kolbe-db-backup-testing/export-20200625-221301.tgz)
      if find /data -name metadata | egrep '.*'; then
          echo "data already exist"
          exit 0
      else
          rclone --config /etc/rclone/rclone.conf copy -P s3://kolbe-db-backup-testing/export-20200625-221301.tgz /data
          cd /data && tar xzvf ${filename}
      fi

Describe the feature you'd like:

Maybe just remove the "z" from the tar command so it auto-detects the compression algorithm?

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the tidb-cloud-backup init container and the /bin/sh command shown in the issue, including where the pingcap/tidb-cloud-backup image is built. Check how the archive command handles compression, then verify that the init container successfully retrieves and extracts non-gzip archives without breaking gzip support.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, shell
Domain
cloud, devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.