💡 feat: expose Test Engine runs and failed executions in the CLI
- Dominant language
- Go
- Stars
- 207
- Forks
- 70
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 24
Description
### Is your feature request related to a problem?
Test Engine results are not reachable from the CLI. For a pipeline whose failures are mostly
test failures, that is the part of the build you actually want, and it is the one part the CLI
cannot show you.
The gap is visible in what the raw log gives you instead: a test runner's stdout, where
finding which test failed and why means scrolling past everything that passed. Test Engine has
already parsed that into individual executions with names, durations, failure messages and
stack traces.
The official MCP server exposes this — `get_build_test_engine_runs`, `list_tests_for_build`,
`get_failed_executions` (with an expanded form carrying stack traces), `get_test_run` and
`get_test`. The CLI has no equivalent for any of them.
### Describe the solution you'd like.
Enough surface to answer "which tests failed in this build, and why":
- `bk test-run list -p --build ` — the Test Engine runs attached to a build
- `bk test-run failures ` — failed executions with names and failure messages,
`--expanded` for stack traces
- participation in `--max-tokens`, since stack traces are exactly the payload that needs a
budget
Scoping it to failures first would cover most of the value; full test listing and per-test
history matter less from a terminal.
### Describe alternatives you've considered.
Reading the job log and parsing the runner's output. That is what the CLI forces today, and it
is per-framework, brittle, and loses the structure Test Engine already produced.
`bk api` against the Test Engine endpoints works but means hand-constructing paths and
reimplementing the output handling the rest of the CLI provides.
### Additional context
Version `bk 3.55.0`. Related: #963 (a build failure summary would naturally include failed
test executions).
Contributor guide
Research direction
Start by tracing the existing CLI command patterns and output handling, including --max-tokens, then compare them with the MCP operations named in the issue: get_build_test_engine_runs and get_failed_executions. Done means the CLI can list a build's Test Engine runs and show failed executions, with --expanded stack traces and token-budget support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100