requires: declare commands that must resolve, and report where each landed
- Dominant language
- TypeScript
- Stars
- 13
- Forks
- 1
- Avg merge
- 3h 38m
- Merged PRs (30d)
- 3
Description
tryscript resolves a command through `PATH` and says nothing about where it landed. For a
golden suite, that is the difference between a proven result and a green check.
A golden test's whole claim is that a *particular* program produced *particular* bytes.
`path:` prepends to the inherited `PATH` rather than replacing it, so if a declared entry
fails to resolve — variable unset, build directory cleaned, a typo, a cross-compiled
layout — lookup continues into the inherited `PATH` and finds whatever is installed there.
The suite then passes while exercising a build nobody selected, with no diagnostic.
That is not hypothetical. It happened in `fdu`: every golden selected its build with
`path: - $TRYSCRIPT_GIT_ROOT/target/debug`, and whenever that failed to resolve the
sessions silently fell through to `~/.cargo/bin/fdu` — the developer's *installed* copy —
and passed.
## Proposal
```yaml
requires:
- fdu
```
Named commands must resolve before the first session runs. If one does not, abort with a
message naming the command and the directories searched, rather than letting a hundred
confusing diffs appear later.
And report where each landed:
```
resolved fdu -> /…/target/debug/fdu (12 files, 129 sessions)
```
The reporting half is the one that matters. It makes a run legible rather than merely
correct — the same reason a good test harness says what it ran, not just whether it
passed. Today the only way to know which binary a suite exercised is to reason about
`PATH` composition and hope.
## Notes
- Lookup should respect `PATHEXT` on Windows rather than assuming a bare name.
- Resolution should use the same composed `PATH` the sessions get, including `path:`
entries, so it answers the question the sessions will actually ask.
- Worth failing rather than warning: a suite that runs without the program it is about
does not produce a weaker result, it produces a meaningless one.
## Context
`fdu` worked around this by naming the binary's directory through a variable and
preflighting it in an external runner script before invoking tryscript. That works, but it
is machinery every project would otherwise have to build for itself, and it cannot report
what tryscript itself resolved.
Related: jlevy/tryscript#51, which made `env:` expand variables the way `path:` already
did — the same family of "front matter should be able to say what it means about which
program runs".
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing how front matter handles `path:` and how the composed PATH reaches sessions, then inspect the existing `env:` expansion behavior and golden-test runner. Add `requires:` resolution before the first session, honoring `PATHEXT` and naming searched directories on failure. Report each resolved command, its path, file count, and session count so a run shows what it actually exercised.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100