[Feature Request] Show Live Coverage Report while the fuzzer is running
- Dominant language
- Haskell
- Stars
- 3.2k
- Forks
- 432
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 5
Description
### Describe the desired feature
# Preamble
When running echidna our goal is to maximize coverage.
In order to iterate on coverage we have to stop the fuzzer and evaluate which lines are not being hit.
To make this process faster [we use foundry](https://book.getrecon.xyz/writing_invariant_tests/create_chimera_app.html) to test if the coverage could theoretically be hit.
This is still not ideal and we ultimately have to run Echidna many times just to generate a coverage report.
Then we'd have to decide if we need to add clamping / change the handlers, or just let the fuzzer run longer.
If we could see the coverage live, we'd make many edits to the handlers while looking at the coverage and then we'd run the fuzzer again.
# Ask
Have the coverage report update live (ideally via a CLI arg), this would allow devs to quickly be able to identify hard to reach parts of the coverage, which will make the DX much faster.
If the performance loss is too high, it would be completely fine to simply add a shortcut to generate the coverage report on request.
I believe most developers will massively benefit by this as the setup time for Echidna (Compilation + Slither), tends to be one of it's few downsides left.
Contributor guide
Research direction
Start by tracing the fuzzer's coverage-report path and the CLI argument handling mentioned in the request. Compare a live update during execution with an on-request snapshot, then verify that the generated report reflects current coverage and that the performance impact is acceptable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell, solidity
- Domain
- developer-experience, security, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100