Refine enforce gas cost min value threshold behaviour in ``isoltest``
- Dominant language
- C++
- Stars
- 25.7k
- Forks
- 6.2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 29
Description
Currently ``isoltest`` just ignores any existing test expectations, if the actually consumed gas amount is below the ``--enforce-gas-cost-min-value`` value.
For example ``test/libsolidity/semanticTests/externalContracts/snark.sol`` at the time of writing says ``// gas irOptimized: 95261`` for ``verifyTx()``. The actual value currently is ``95259``, but the test passes.
The test also passes if I change the expectation to any arbitrary value (both below or above the "enforce-gas-cost-min-value" threshold).
This behaviour is not particularly nice - it will lead to ghost-test-expectations whenever a case falls below the gas threshold.
The only thing determined by ``--enforce-gas-cost-min-value`` should be to determine at which threshold ``isoltest`` should start to actively *add* gas expectations. Existing expectations should always be checked and kept up to date. [alternative: actively *remove* expectations below the threshold; I prefer always checking, though]
IIRC this was always the intended meaning of the option, but as @cameel pointed out, it's name is not particularly well chosen, so in the process it might make sense to rename it.
Contributor guide
Research direction
Start with isoltest's handling of --enforce-gas-cost-min-value and reproduce the behavior using test/libsolidity/semanticTests/externalContracts/snark.sol, especially verifyTx(). Check that existing gas expectations are still validated below the threshold, while the option controls only when expectations are added; done means the arbitrary 95259/95261 expectation case no longer passes silently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, solidity
- Domain
- compilers, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100