pingcap / pingcap/tidb-operator
Skip rclone in the `tidb-lightning` chart
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.3k
- Forks
- 540
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 18
Description
Feature Request
Is your feature request related to a problem? Please describe:
To import from cloud storage, as of current version (v1.7.0-alpha.3) of TiDB Lightning Helm Chart it relies on rclone to copy those files locally. This was introduced 6 years ago #817, but since then Lightning has supported downloading from cloud storage natively.
The additional rclone pod complicates deployment and credential configuration.
Describe the feature you'd like:
Make it possible to directly use an s3:// / gcs:// etc data source directly, without any volume mount for the data-source
None of the current dataSource works:
localonly accepts an actual path forhostPathremoteis rclone-basedadhocrequires a PVC
Describe alternatives you've considered:
We just give up on the Helm chart and schedule a custom Lightning job.
Teachability, Documentation, Adoption, Migration Strategy:
pingcap/docs-tidb-operator#2727
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with charts/tidb-lightning/values.yaml around the dataSource settings at lines 30-31, then trace the chart templates for the local, remote, and adhoc modes and the rclone pod. Confirm how the chart currently handles cloud sources and volume mounts. Done means an s3://, gcs://, or similar source can be used directly without a data-source volume or rclone pod.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100