runtimeverification / runtimeverification/kontrol
`AssertTest.checkFail_assert_false` consistently failing on CI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 122
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
In KEVM update PR, AssertTest.checkFail_assert_false() consistently failed on expected output comparison in Booster integration tests for several times, but passed locally without producing any output diff. Eventually, it passed on CI too. It'd be good to investigate what caused the failure and how it could be avoided. Some examples of the failing CI job are
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with AssertTest.checkFail_assert_false() in the Booster integration tests and compare its expected-output behavior with the cited CI jobs and the KEVM update PR. Reproduce the intermittent failure in CI, determine why the output comparison differs without a diff, and verify that the cause is addressed by rerunning the affected tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100