[br] BR abort can delete a live restore because its row lock suppresses heartbeats
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
> Evidence status: confirmed.
### 1. Minimal reproduce step (Required)
On current source, create a running restore registry entry and continuously call UpdateHeartbeat from its dedicated heartbeat session. Concurrently call FindAndDeleteMatchingTask for the same configuration. A deterministic real-TiKV test shortens only the one-minute ticker and five-check count. Before the abort transaction, heartbeat advances and the task is classified active. After abort locks the row with SELECT FOR UPDATE, heartbeat UPDATE gets kv:9007 write conflict; both observations stay unchanged; abort returns the task ID and the row count becomes zero. This RED repeated 3/3. A no-heartbeat stale control deleted the row successfully 3/3.
### 2. What did you expect to see? (Required)
A matching restore with an active heartbeat writer must not be deleted; abort should return zero and retain the registry row.
### 3. What did you see instead? (Required)
The abort transaction holds the matching registry row lock while waiting for heartbeat changes. That same lock prevents the independent heartbeat session from updating, so the observer manufactures the stale evidence it relies on and then deletes the live task.
### 4. What is your TiDB version? (Required)
current master 13282a8bd06b
Likely root cause and fix direction
FindAndDeleteMatchingTask starts one pessimistic transaction, locks the row with FOR UPDATE, and calls isTaskStale inside that transaction. UpdateHeartbeat writes last_heartbeat_time on the same row through another session and conflicts with the abort lock. After the unchanged window, deletion uses only task ID.
Contributor guide
Research direction
Start by locating FindAndDeleteMatchingTask and UpdateHeartbeat, then run the deterministic real-TiKV reproduction described in the issue. Trace the transaction and row-lock behavior around isTaskStale. Done means an active heartbeat prevents deletion and returns zero, while a genuinely stale task is still deleted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100