proxy: do not do flush if the coming CompactLog is doomed to fail
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Enhancement
exec_compact_log can fail for the following reasons which may be common in some TiFlash's cases.
...
if compact_index <= first_index {
debug!(
"compact index <= first index, no need to compact";
"region_id" => self.region_id(),
"peer_id" => self.id(),
"compact_index" => compact_index,
"first_index" => first_index,
);
return Ok((resp, ApplyResult::None));
}
if self.is_merging {
info!(
"in merging mode, skip compact";
"region_id" => self.region_id(),
"peer_id" => self.id(),
"compact_index" => compact_index
);
return Ok((resp, ApplyResult::None));
}
...
and
else if compact_index > state.get_applied_index() {
return Err(box_err!(
"compact index {} > applied index {}",
compact_index,
state.get_applied_index()
));
}
We can detect them in advance, and avoid them in advance, by returning false in pre_exec.
Among the 3 cases above, case 1 is more common. Consider the following sequence
apply snap -> compact log -> apply snap
The second apply snap may happen before exec compact log.
After the second apply snap, the truncated index can be updated to greater than the first compact log.
Also consider the following sequence
propose compact log 1 -> propose compact log 2 -> exec compact log 1 -> exec compact log 2(compact index <= first index)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the proxy's pre_exec and exec_compact_log paths, then trace the apply-snapshot and successive compact-log sequences described in the issue. Identify how the three failure cases can be detected before execution. Done means pre_exec returns false for compact logs that are already doomed to fail, without performing the unnecessary flush.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100