runtimeverification / runtimeverification/kontrol

`kontrol show` crashes if more than one test match the prefix

Open
#363 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug cli
Dominant language
Python
Stars
122
Forks
16
PR merge metrics
No merged PRs in 30d

Description

If kontrol show is called with a prefix that matches more than one test, it crashes with ValueError: Expected a single element, found more. For example, if there are two tests, one called StartPrankTestSymbolic.testSymbolicPrank and another called StartPrankTestSymbolic.testSymbolicPrankSplit, calling kontrol show StartPrankTestSymbolic.testSymbolicPrank produces the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/lucasmt/kontrol/refute-node/src/kontrol/__main__.py", line 90, in main
    execute(**vars(args))
  File "/home/lucasmt/kontrol/refute-node/src/kontrol/__main__.py", line 336, in exec_show
    output = foundry_show(
  File "/home/lucasmt/kontrol/refute-node/src/kontrol/foundry.py", line 522, in foundry_show
    contract_name, _ = single(foundry.matching_tests([test])).split('.')
  File "/home/lucasmt/pyk/refute-node/src/pyk/utils.py", line 272, in single
    raise ValueError('Expected a single element, found more', fst, snd)
ValueError: ('Expected a single element, found more', 'test%StartPrankTestSymbolic.testSymbolicPrankSplit(address)', 'test%StartPrankTestSymbolic.testSymbolicPrank(address)')

Note that kontrol show StartPrankTestSymbolic.testSymbolicPrankSplit can be called with no problem, but there is no way to display the KCFG for StartPrankTestSymbolic.testSymbolicPrank, since its name is a strict prefix of the other test.

Some alternatives on how to handle this:

  • Require the argument to kontrol show to be the full name of the test, not just a prefix.
  • Keep the current behavior by default, but add an option (e.g. --full-name) to match the full name instead. Rewrite the error message above to suggest this as an option to the user.
  • Print the KCFGs for all tests that match the prefix (might be hard to read and confusing if the KCFGs are large).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/kontrol/foundry.py at foundry_show and follow the call from exec_show in src/kontrol/main.py. Reproduce the prefix collision with the two test names in the report and inspect the matching behavior that reaches pyk/utils.py. Done means kontrol show handles a test name that is a strict prefix without raising the reported ValueError, according to the selected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.