/ship Step 5 hardcodes `bin/test-lane` + `npm run test` regardless of detected framework
- Dominant language
- TypeScript
- Stars
- 133k
- Forks
- 19.9k
- Avg merge
- 18h 46m
- Merged PRs (30d)
- 26
Description
## What happened
Ran `/ship` in a Django-only project. Step 4's `{{TEST_BOOTSTRAP}}` correctly detected the runtime:
> Test framework detected: Python/Django with chores/tests.py. Skipping bootstrap.
But Step 5 then told the agent to run:
```bash
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
wait
```
Neither command exists in a Django project. Both are hardcoded — `bin/test-lane` is Rails-specific and `npm run test` is Node-specific. They appear to have leaked in from a Rails-with-Vitest project the skill was authored against.
## Where
`ship/SKILL.md.tmpl:148-160` — Step 5 runs both hardcoded commands unconditionally, without regard to what Step 4 just detected.
## Why it's a bug
This directly violates the project's own rule in `CLAUDE.md`:
> ## Platform-agnostic design
>
> Skills must NEVER hardcode framework-specific commands, file patterns, or directory structures. Instead:
> 1. Read CLAUDE.md for project-specific config (test commands, eval commands, etc.)
> 2. If missing, AskUserQuestion — let the user tell you or let gstack search the repo
> 3. Persist the answer to CLAUDE.md so we never have to ask again
Step 4 already follows this pattern (detects runtime, bootstraps if needed). Step 5 doesn't — it just assumes Rails + Vitest.
## Suggested fix shape
1. Read the test command from the project's `CLAUDE.md` (the `## Testing` section that the bootstrap flow writes). If present, run it.
2. If missing, `AskUserQuestion` and persist the answer back to `CLAUDE.md` so the next `/ship` doesn't re-ask.
3. Drop the hardcoded `bin/test-lane` + `npm run test` pair.
4. Guardrail: add a test that renders `ship/SKILL.md` and fails if it contains hardcoded framework commands outside a detection block, so the regression can't quietly come back.
## Repro
Run `/ship` in any non-Rails, non-Node project (Django, Go, Rust, etc). Step 5 will propose `bin/test-lane` + `npm run test`.
## Out of scope
Step 6 ("Eval Suites") also has Rails-specific path patterns and `bin/test-lane --eval` commands, but it self-skips cleanly on non-matching diffs ("No prompt-related files changed — skipping."), so it's cosmetic for now. Leaving that for a separate issue if it ever actually fires incorrectly.
Contributor guide
Assessment
This issue has not been assessed yet.