benchmark.sh: add missing commands for pre-release pipeline
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 81.1k
- Forks
- 5.1k
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 35
Description
Problem
scripts/benchmark.sh currently tests 22 commands but is missing ~12 testable commands that exist in RTK. This means regressions in those filters can slip through the pre-release pipeline undetected.
Missing commands (testable)
| Command | Notes |
|---|---|
rtk cargo build/test/clippy |
Always available (we're a Rust project) |
rtk smart <file> |
2-line summary, easy to test |
rtk diff <f1> <f2> |
File diff, easy to test with temp files |
rtk tree |
Directory tree, easy to test |
rtk curl <url> |
HTTP fetch, test with public URL |
rtk wget <url> |
Download, test with public URL |
rtk npm run |
If package.json present |
rtk npx |
If package.json present |
rtk proxy <cmd> |
Passthrough, always available |
rtk ruff check |
Already has Python fixture, just uses wrong $RTK test wrapper |
rtk pytest |
Same — should use $RTK pytest not $RTK test pytest |
rtk pip list |
If pip available |
rtk go test/build |
Already has Go fixture, just uses wrong $RTK test wrapper |
rtk golangci-lint |
Same — should use $RTK golangci-lint not $RTK test golangci-lint |
rtk format |
Universal format checker |
Not benchmarkable (skip)
gain, discover, learn, config, init, cc-economics, hook-audit — these are analytics/setup commands, not filters.
Current bugs in benchmark.sh
The Python and Go sections use $RTK test ruff check . and $RTK test golangci-lint run instead of the dedicated $RTK ruff check . and $RTK golangci-lint run commands. This means we're benchmarking the generic test runner instead of the specialized filters.
Expected outcome
All filter commands covered in benchmark.sh so we can catch token savings regressions before release.
# Pre-release validation
cargo build --release
bash scripts/benchmark.sh # Should test ALL filters
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review scripts/benchmark.sh and its existing Python and Go fixture sections; start by running cargo build --release and bash scripts/benchmark.sh to see the current coverage. Add benchmark coverage for the listed testable filters and correct the specialized Python and Go invocations; done means the script exercises all filter commands and completes successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, rust
- Domain
- cli, release, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100