apple / apple/foundationdb

Spurious Joshua timeout in shared environment

Open
#4,116 2 comments 0 reactions 0 assignees View on GitHub
testing
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

We observed that running in a shared environment (e.g., docker and AWS spot instances), the correctness runs of Joshua results are often `timeout` errors. These errors can become very noisy if the CPU resources on the test machine is heavily contended. So it would be very useful we can filter them out.

One idea I have is for `TestHarness` to check the progress of simulation runs. Rerun the simulation for `timeout` errors.

Contributor guide

Open the contributing guide

Research direction

Locate TestHarness and the Joshua correctness-run handling; first inspect how simulation progress and timeout errors are reported. Done means timeout results caused by shared-resource contention are handled by rerunning the simulation, with coverage for that behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, cpp, docker
Domain
distributed-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.