KaykCaputo / KaykCaputo/oracletrace

[Feature]: Support --repeat aggregation in oracletrace run subcommand

Open
#67 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted important
Dominant language
Python
Stars
22
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Summary

Implement median-based repeat aggregation for `oracletrace run -- pytest`, matching the behavior already available when tracing a script directly.

## Problem

`--repeat` is defined on `_BASE_PARSER` (shared by both entry points) but aggregation logic exists only in `_run_target()`. When using `oracletrace run --repeat 3 -- pytest`, pytest runs once and repeat is silently ignored.

## Proposed Solution

1. Extract the repeat loop + `FunctionAggregate` median logic from `_run_target()` into a shared helper (e.g. `_aggregate_repeated_runs()`)
2. Update `_run_pytest()` to invoke pytest N times when `--repeat > 1` and aggregate trace data
3. Preserve current semantics: skip `show_results` when `runs > 1` (same as script mode)
4. Define exit-code precedence when combining `--repeat`, pytest failures, and `--fail-on-regression`
5. Add tests and update `docs/docs/cli-reference.md`

## Use Case

CI pipelines using `oracletrace run -- pytest tests/` need median aggregation to reduce OS-level noise — the primary motivation for `--repeat` in v3.0.0. Without this, CI users cannot benefit from repeat when tracing test suites.

## Example (optional)

```bash
oracletrace run --repeat 5 --json current.json --compare baseline.json --fail-on-regression -- pytest tests/ -q
```

```text
# pytest runs 5 times; exported/compared trace uses median total_time and call_count per function
```

## Alternatives Considered

- Document that `--repeat` is script-only and reject it on `run` — simpler but worse UX since the flag is already accepted.
- Require users to shell-loop pytest manually — workable but duplicates logic already in `_run_target()`.

## Additional Context

Verified locally: `oracletrace run --repeat 3 -- pytest` calls `pytest.main()` once.

Relevant files:
- `oracletrace/cli.py` — `_run_target()` (lines 227–253), `_run_pytest()` (lines 297–318)
- `oracletrace/tracer.py` — `FunctionAggregate`
- `tests/test_cli.py` — existing `--repeat` and `run pytest` tests
- `docs/docs/cli-reference.md` — CI pytest workflow section

## Checklist

- [x] I searched existing issues before opening this request
- [x] I described the problem and why this feature is useful
- [x] I provided enough detail for implementation discussion

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.