feat(verify): attest that every declared capability had a check that ran
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8
- Forks
- 1
- Avg merge
- 18h 23m
- Merged PRs (30d)
- 63
Description
Skip classification catches a check that ran and gave up. It cannot catch a capability that had no check at all — which is #602 exactly: test_workbench_login skips under SSO and no other lane verifies a Workbench login, so nothing is even present to classify.
Derive the expected checks from what vip.toml declares. If Connect is configured with SSO, the run must name a check that actually exercised SSO login; if none executed, report the capability as unproven rather than staying silent. This is what turns the report into "31 of your 47 declared capabilities were verified" instead of a pass/fail count over whatever happened to run.
Depends on the unproven outcome from #617, which already supplies the reporting and exit-code machinery.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the capability declarations in vip.toml and the test_workbench_login check, then review the unproven outcome from #617. Trace how configured SSO capabilities are matched to checks that actually ran. Done means a declared capability with no executed check is reported as unproven, with the existing reporting and exit-code behavior preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100