KaykCaputo / KaykCaputo/oracletrace

[Feature]: Universal run command — trace any Python invocation, not just pytest

Open
#81 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
22
Forks
16
PR merge metrics
No merged PRs in 30d

Description

Summary
Generalize oracletrace run to trace any Python invocation (python -m , python script.py ), removing the hardcoded pytest-only restriction, while keeping tracing in-process so sys.setprofile() still captures the run.

Problem
oracletrace run currently rejects everything except pytest. In [_run_command](https://github.com/KaykCaputo/oracletrace/blob/master/oracletrace/cli.py) the command is checked with if cmd == "pytest" and anything else exits with error: unsupported command ''. Only 'pytest' is supported. This blocks unittest, Django's manage.py test, custom entrypoints, and plain python -m ... runs — even though the run --help text and docs imply general command support.

Proposed Solution
Replace the pytest-only branch with a general in-process dispatcher for Python invocations:

python -m [args] → run via runpy.run_module(module, run_name="__main__")
python [args] → run via runpy.run_path(script, run_name="__main__")
Set sys.argv to the child args for the duration of the run so argparse-based programs work.
Keep pytest working (it becomes the -m pytest case; the existing fast path can stay).
Tracing stays in-process, so the existing sys.setprofile() engine captures everything with no change to the tracer.

Use Case
CI performance gating for any Python test runner or entrypoint, not just pytest:

oracletrace run --compare baseline.json --fail-on-regression -- python -m unittest
oracletrace run -- python manage.py test
oracletrace run -- python script.py --arg1 val
Example

oracletrace run -- python -m unittest discover tests/

HTML report generated: /path/report.html
Build failed: performance regression above 5.00% detected.
Alternatives Considered
subprocess.run() wrapping the command — rejected. A child process is not visible to the parent's sys.setprofile(), so traces would come back empty. In-process runpy keeps parity with the existing script-tracing path (_run_target already uses runpy.run_path).
Bootstrap injection (a sitecustomize/PYTHONSTARTUP shim that installs the tracer in the child) — would enable tracing arbitrary non-Python-launched binaries, but is a larger design. Out of scope here; could be a follow-up for full "any command" support.
Additional Context
Relevant code: _run_command and the cmd == "pytest" gate, _run_pytest, and the existing in-process runpy usage in _run_target. Happy to open a PR implementing the -m module and script.py paths with tests if this direction looks good.

Checklist
I searched existing issues before opening this request
I described the problem and why this feature is useful
I provided enough detail for implementation discussion

Contributor guide

Open the contributing guide

Research direction

Start in oracletrace/cli.py at _run_command and the cmd == "pytest" gate, then compare _run_pytest with the existing runpy usage in _run_target. Verify that python -m modules and script.py invocations run in-process with child arguments in sys.argv, while pytest continues to work and tracing remains captured.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, devtools
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.