jlevy / jlevy/tryscript

requires: declare commands that must resolve, and report where each landed

Offen
#54 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
TypeScript
Sterne
13
Forks
1
Ø Merge
3 Std. 38 Min.
Gemergte PRs (30 T.)
3

Beschreibung

tryscript resolves a command through `PATH` and says nothing about where it landed. For a
golden suite, that is the difference between a proven result and a green check.

A golden test's whole claim is that a *particular* program produced *particular* bytes.
`path:` prepends to the inherited `PATH` rather than replacing it, so if a declared entry
fails to resolve — variable unset, build directory cleaned, a typo, a cross-compiled
layout — lookup continues into the inherited `PATH` and finds whatever is installed there.
The suite then passes while exercising a build nobody selected, with no diagnostic.

That is not hypothetical. It happened in `fdu`: every golden selected its build with
`path: - $TRYSCRIPT_GIT_ROOT/target/debug`, and whenever that failed to resolve the
sessions silently fell through to `~/.cargo/bin/fdu` — the developer's *installed* copy —
and passed.

## Proposal

```yaml
requires:
- fdu
```

Named commands must resolve before the first session runs. If one does not, abort with a
message naming the command and the directories searched, rather than letting a hundred
confusing diffs appear later.

And report where each landed:

```
resolved fdu -> /…/target/debug/fdu (12 files, 129 sessions)
```

The reporting half is the one that matters. It makes a run legible rather than merely
correct — the same reason a good test harness says what it ran, not just whether it
passed. Today the only way to know which binary a suite exercised is to reason about
`PATH` composition and hope.

## Notes

- Lookup should respect `PATHEXT` on Windows rather than assuming a bare name.
- Resolution should use the same composed `PATH` the sessions get, including `path:`
entries, so it answers the question the sessions will actually ask.
- Worth failing rather than warning: a suite that runs without the program it is about
does not produce a weaker result, it produces a meaningless one.

## Context

`fdu` worked around this by naming the binary's directory through a variable and
preflighting it in an external runner script before invoking tryscript. That works, but it
is machinery every project would otherwise have to build for itself, and it cannot report
what tryscript itself resolved.

Related: jlevy/tryscript#51, which made `env:` expand variables the way `path:` already
did — the same family of "front matter should be able to say what it means about which
program runs".

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Beginne damit nachzuverfolgen, wie front matter `path:` verarbeitet und wie der zusammengesetzte PATH an Sessions weitergegeben wird, und untersuche dann das bestehende Verhalten der `env:`-Erweiterung und den Golden-Test-Runner. Füge die Auflösung von `requires:` vor der ersten Session hinzu, berücksichtige `PATHEXT` und nenne bei einem Fehlschlag die durchsuchten Verzeichnisse. Melde für jeden aufgelösten Befehl seinen Pfad, die Dateianzahl und die Sitzungsanzahl, damit ein Lauf zeigt, was tatsächlich ausgeführt wurde.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
typescript
Bereich
cli, testing-qa
Issue-Typ
Feature
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Aktiv
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
52/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.