garrytan / garrytan/gstack

gstack-model-benchmark accepts malformed --timeout-ms values

Open Beginner friendly
#1,726 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

## Observed problem

On current upstream `main` (`cf50443b63e461a7c0796857f69d572781acab8e`), `gstack-model-benchmark --timeout-ms` is parsed with `parseInt`, so malformed values are accepted instead of rejected.

Repro:

```bash
bun run bin/gstack-model-benchmark --prompt hi --timeout-ms 1abc --dry-run
bun run bin/gstack-model-benchmark --prompt hi --timeout-ms nope --dry-run
bun run bin/gstack-model-benchmark --prompt hi --timeout-ms 0 --dry-run
```

Current output includes:

```text
timeout_ms: 1
timeout_ms: NaN
timeout_ms: 0
```

## Expected behavior

`--timeout-ms` should require a positive integer timeout and fail fast for malformed, non-numeric, zero, negative, or decimal values before dry-run or provider execution.

## Duplicate searches performed

I checked for existing open issues and PRs using:

- `gstack-model-benchmark --timeout-ms`
- `benchmark timeout parseInt malformed --timeout-ms`
- `bin/gstack-model-benchmark timeout`
- `timeout_ms NaN benchmark`

No duplicate issue or canonical open PR covered this behavior. Open PR #1495 also touches `bin/gstack-model-benchmark`, but it is an additive Ollama provider change and does not validate `--timeout-ms`.

## Candidate fix shape

Add a strict positive-integer parser for `--timeout-ms`, emit a concise CLI error for invalid values, and cover dry-run behavior in `test/benchmark-cli.test.ts`.

Contributor guide

Open the contributing guide

Research direction

Start in bin/gstack-model-benchmark and inspect how --timeout-ms is parsed. Read the related cases in test/benchmark-cli.test.ts, then run the benchmark CLI tests with malformed, zero, negative, and decimal values. Done means invalid values produce a concise error before dry-run or provider execution, while positive integers remain accepted.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
cli, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.