Changefeed lag increase continously with frequent truncate tables
Open
Nobody has claimed this yet.
ga-optional
type/question
- Dominant language
- Go
- Stars
- 56
- Forks
- 63
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
A small number of table truncations will not cause lag fluctuations.
However, extremely frequent truncation (e.g., 250 truncations per minute) can cause lag fluctuations.
The specific operation is to create 100 tables, randomly truncate 30 of them, then sleep for 1 second, and repeat this process.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the described workload: create 100 tables, repeatedly truncate 30 random tables, and sleep for one second while observing changefeed lag. No source file or test is named; investigate the truncate and lag-handling paths, and consider the issue addressed when frequent truncations no longer cause lag to increase continuously.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100