runtimeverification / runtimeverification/kontrol

Optimize Foundry test suite

Open
#197 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
122
Forks
16
PR merge metrics
No merged PRs in 30d

Description

The Foundry test suite we use for CI proofs must be better organized. With duplicated test contracts (such as PlainPrankTest.t.sol and PrankTest.t.sol), contracts that do not use the setUp functionality repeatedly deploy auxiliary contracts in the tests. In addition, a lot of tests are skipped to keep the CI time low.

A few optimizations:

  • Get a smaller test-suite codebase, resulting in lower kompile time:

    • Merge duplicated contracts
    • Remove/merge tests with duplicated logic
  • Improve performance of the tests:

    • Use the setUp functionality instead of deploying auxiliary contracts in the test.
    • replace assertEq and assertGt assertion calls with assert. Since assertEq is a function from the forge test library, this will reduce the number of kcfg nodes generated by removing calls to the forge test contract. We can add individual unit tests to check that assertEq and assert act similarly.
    • use notBuiltinAddress() in unit tests that use symbolic addresses to avoid unnecesary branchings.

By implementing these we could (1) reduce CI time and/or (2) run more tests that are currently skipped in the same amount of time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the Foundry test suite, especially PlainPrankTest.t.sol and PrankTest.t.sol, then identify duplicated test logic and auxiliary-contract deployments. Review the skipped tests and uses of setUp, assertEq, assertGt, and notBuiltinAddress. Done means a smaller, faster suite with fewer skipped tests while retaining equivalent assertion coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
solidity
Domain
ci-cd, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.