apple / apple/foundationdb

Degraded transaction logs are not removed during recovery

Open
#5,679 3 comments 0 reactions 1 assignee Claimed by @halfprice View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

When a transaction log is unable to commit or do a few other things during its local recovery, it gets marked degraded. This status gets reported to the cluster controller, who would then attempt to recruit a new transaction subsystem without any degraded logs.

If a log gets reported degraded during recovery, though, and that degradation prevents the recovery from completing, then the cluster controller will not try to replace it. If I understand correctly, this is because `betterMasterExists` does not attempt to reevaluate the cluster layout if it is not sufficiently recovered:

https://github.com/apple/foundationdb/blob/5a5f724d9c7f1c1fac47a610264effc4b44d300e/fdbserver/ClusterController.actor.cpp#L2223

This behavior was observed in 6.2, and while the line above still exists I'm not sure if this is impacted by some of the other newer changes to the degradation logic.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.