KaykCaputo / KaykCaputo/oracletrace
[Feature]: Retrieve baseline automatically from branch or commit
- Dominant language
- Python
- Stars
- 22
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Add support for automatically retrieving a baseline trace from a Git branch or a specific Git commit.
Proposed flags:
```bash
oracletrace app.py --get-baseline-from-branch main
oracletrace app.py --get-baseline-from-commit a1b2c3d
```
## Problem
Today, performance comparisons require manually generating, storing, and providing baseline JSON files.
Typical CI workflows require additional scripting to:
1. Checkout a reference branch or commit.
2. Generate a baseline trace.
3. Store the generated JSON artifact.
4. Restore it later for comparison.
This introduces operational overhead and makes regression detection harder to adopt.
## Proposed Solution
Allow OracleTrace to automatically obtain a baseline from Git history.
### Branch mode
```bash
oracletrace app.py --get-baseline-from-branch main
```
Workflow:
1. Resolve the latest commit from the specified branch.
2. Locate a previously generated OracleTrace baseline associated with that commit.
3. Use it as the comparison baseline.
### Commit mode
```bash
oracletrace app.py --get-baseline-from-commit a1b2c3d
```
Workflow:
1. Resolve the specified commit.
2. Locate a baseline associated with that commit.
3. Use it for comparison.
Internally this could integrate with:
- Local cache directory
- CI artifacts
- Future remote baseline storage backends
## Example
Current workflow:
```bash
oracletrace app.py --json baseline.json
oracletrace app.py \
--json current.json \
--compare baseline.json
```
Desired workflow:
```bash
oracletrace app.py \
--get-baseline-from-branch main \
--fail-on-regression \
--threshold 10
```
or
```bash
oracletrace app.py \
--get-baseline-from-commit a1b2c3d \
--fail-on-regression \
--threshold 10
```
## Use Case
### Pull Request validation
A CI pipeline could automatically compare a feature branch against the latest baseline generated on `main`.
```bash
oracletrace app.py \
--get-baseline-from-branch main \
--fail-on-regression
```
### Historical regression investigation
Developers could compare performance against a known-good commit.
```bash
oracletrace app.py \
--get-baseline-from-commit a1b2c3d
```
## Alternatives Considered
- Continue managing baseline files manually.
- Store baseline paths in CI configuration.
- Use external scripts to fetch historical baselines before running OracleTrace.
## Open Questions
- Where should baselines be stored and indexed?
- Local cache
- GitHub Actions artifacts
- Dedicated remote storage
- Should OracleTrace generate the baseline automatically if one does not exist?
- How should branch names map to baseline artifacts?
## Additional Context
This feature would significantly improve CI/CD integration by reducing manual baseline management and enabling Git-native performance comparisons.
## Checklist
- [x] I searched existing issues before opening this request
- [x] I described the problem and why this feature is useful
- [x] I provided enough detail for implementation discussion
Contributor guide
Assessment
This issue has not been assessed yet.