TiFlash failed to send store heartbeat to pd while alive
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
One tiflash node become "disconnect store", which means PD does not receive store heartbeat from the tiflash node for more than 20 seconds.
By default, every 10 second TiFlash/TiKV send store heartbeat to PD. And according to the metrics, TiFlash did not send store heartbeat to PD at that time
Maybe related with massive remove-peer requests
2. What did you expect to see? (Required)
3. What did you see instead (Required)
4. What is your TiFlash version? (Required)
v6.5.6
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is named. Start by tracing TiFlash's store-heartbeat path to PD and the metrics showing missed heartbeats, then examine whether massive remove-peer requests coincide with the gap. Done means the reported alive TiFlash node continues sending store heartbeats and the failure is covered by an appropriate regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100