argotorg / argotorg/solidity

Improve reviewability of extracted tests (for SMTChecker tests and source locations in general).

Open
#9,581 15 comments 1 reaction 0 assignees View on GitHub
smt testing :hammer:
Dominant language
C++
Stars
25.7k
Forks
6.2k
Avg merge
1d 11h
Merged PRs (30d)
21

Description

Especially for the SMTChecker tests reviews would be much simpler, if one could check out the branch locally, build isoltest and have it run all tests touched in the diff from develop. In this mode ``isoltest`` should single-step through the test expectations and highlight each one (even if succeesding) in the source with full error message (including secondary location - although we might add those to the expectations anyways).

If we'd want to be fancy, this mode in isoltest could even have the option to either agree with the test or open an editor, in which a message could be typed that would automatically be added as a comment to the PR (that *should* work using the github command line tools) - but even without that, it'd be a big help.

(ping @leonardoalt for comments)

Contributor guide

Open the contributing guide

Research direction

Start by building isoltest locally and reviewing how it discovers SMTChecker tests and compares a branch with develop. Define the expected review flow for tests touched in the diff, including single-stepping expectations and showing full error messages with secondary locations. Done means isoltest can run this review mode reliably; the optional editor and GitHub CLI integration are explicitly secondary.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
devtools, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.