influxdata / influxdata/influxdb
Improve resilience of `influxd inspect export-lp` when running against a live DB
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
The 2.x implementation of export-lp introduced in #20467 doesn't handle the edge cases of running against a live DB very well. The tool lists the contents of the TSM and WAL directories for the target bucket to find the files it should process. If a file is deleted between that point and the point when the tool actually tries to process it, a warning is logged and the export continues. This matches the 1.x implementation, but (I think) it can result in data being dropped from the export if compactions/snapshots are running concurrently.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the 2.x export-lp implementation introduced in #20467, focusing on how it lists and processes the target bucket's TSM and WAL files. Investigate what happens when compactions or snapshots delete files between listing and processing; done means a live-DB export does not silently drop data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100