apple / apple/foundationdb

QuietDatabase should monitor progress rather than have a hard timeout

Open
#7,920 0 comments 0 reactions 1 assignee Claimed by @sfc-gh-bvr View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

https://github.com/apple/foundationdb/issues/7273 is a case where QuietDatabase times out because the DD queue size hasn't dropped to 0 in the allotted time.
In reality, DD is making progress, but the happens to be a lot of work for DD to do because buggification is enabled and that makes it so that the shard sizes limits are set very low for some seeds. This results in a lot of splits for workloads that do sequential inserts. Those splits result in data movement.
The problem is suppressed by extending the timeout when buggification is on, but those timeouts may also not suffice under certain conditions. Ideally, QuietDatabase should be happy to give DD more time to finish as long as it is making progress, possibly up to some larger hard timeout.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.