micronaut-projects / micronaut-projects/micronaut-http-benchmarks
Upstream subprocess-safe Hyperfoil standalone CLI execution
- Dominant language
- Java
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Goal
Make the generated endpoint smoke benchmarks executable through stock Hyperfoil standalone CLI without maintaining project-specific CLI bootstrap or result-verifier classes.
This repository should ultimately be able to invoke a command similar to:
```bash
run.sh \
--standalone-root "$tmp/controller" \
--timeout 10m \
--fail-on-errors \
--json-result "$tmp/result.json" \
"$tmp/smoke.yaml"
```
## Motivation
Hyperfoil 0.29.3's one-shot `run.sh` is not a reliable subprocess interface for this use case:
- `LoadAndRun.main()` discards the result returned by `exec()`, so failures are not reliably reflected in the process exit status.
- The one-shot command does not expose the standalone controller root, despite `StartLocal` supporting one.
- `wait` has no timeout and can wait indefinitely.
- Runtime and validation errors are printed but do not necessarily fail the command.
- Machine-readable final results are only available indirectly.
Current Hyperfoil main already has a `BaseStandaloneCommand.startLocalCommand()` hook, making the root-directory change smaller than on 0.29.3.
## Proposed upstream work
- Change `LoadAndRun.main()` to propagate the result through `System.exit(...)`.
- Add `--standalone-root ` to the one-shot command and pass it to `start-local`.
- Add a bounded `--timeout ` option to `wait`/one-shot execution.
- Add `--fail-on-errors` so runtime, request-validation, and abnormal termination errors produce a non-zero exit status.
- Add optional `--json-result ` output, preferably reusing Hyperfoil's existing JSON export support.
- Preserve existing interactive behavior by making strict failure and timeout behavior opt-in where necessary.
- Add upstream CLI tests covering exit status, timeout, explicit root, cleanup, runtime errors, and JSON output.
## Repository integration
After the upstream changes are released:
- Update the pinned Hyperfoil version.
- Remove any local standalone CLI bootstrap and standalone result-verifier implementation.
- Invoke stock `run.sh` over the controller SSH connection.
- Keep the external process watchdog as a final infrastructure safeguard.
- Preserve temporary-file and mTLS keystore cleanup.
## Acceptance criteria
- A malformed benchmark exits non-zero.
- A runtime or response-validation failure exits non-zero.
- A timed-out run exits non-zero and cleans up the embedded controller.
- An explicit standalone root isolates persisted data for each invocation.
- A successful run exits zero and optionally writes stable machine-readable results.
- `micronaut-http-benchmarks` no longer packages custom Hyperfoil CLI or result-verifier classes.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with Hyperfoil's LoadAndRun.main(), BaseStandaloneCommand.startLocalCommand(), and the existing start-local/wait command paths. Trace how one-shot execution reports results, handles timeout and cleanup, and exports JSON, then inspect this repository's generated benchmark invocation and custom bootstrap/verifier classes. Done means upstream CLI tests cover exit status, timeout, cleanup, explicit roots, runtime errors, and JSON output, followed by the repository integration changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, shell
- Domain
- cli, testing, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100