apache / apache/otava

Fix branch interpolation logic in regressions command

Open
#118 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
177
Forks
35
Avg merge
3d 16h
Merged PRs (30d)
6

Description

The `otava regressions` command supports an optional `--branch` argument, which specifies the feature branch to compare against the base branch.

## Problem

The current implementation has several limitations:

1. The feature-branch logic is implemented at the importer level, which forces every importer to either implement it or explicitly raise `NotImplementedError` (as is currently the case).
2. The Graphite importer relies on custom interpolation logic for the `$BRANCH` environment variable.
3. There is no way to configure the base branch name.

These limitations lead to several undesirable consequences:

1. To configure a custom base branch, we we must manually expand the `$BRANCH` environment variable (via `expandvars`). PostgreSQL example and e2e test illustrate this problem.
2. Feature-branch testing is not supported by importers other than Graphite.
3. Importer behavior is inconsistent and difficult to reason about.

## Proposed Solution

### CLI API Changes

Replace the existing `--branch` argument with two explicit arguments:

- `--base-branch`
Specifies the base branch to compare against.
Default: `main`

- `--feature-branch`
Specifies the feature branch to compare.
No default.

**Usage semantics:**

- If only `--base-branch` is provided, it overrides the default base branch and tests for regressions within that branch.
- If both `--base-branch` and `--feature-branch` are provided, Otava tests the feature branch for regressions relative to the base branch.
- If only `--feature-branch` is provided, Otava compares the feature branch against the default base branch (`main`).

### Configuration Interpretation Changes

1. **`regressions` command**
Otava expands the `$BRANCH` variable using both the base and feature branch values.
2. **`analysis` command**
Otava expands `$BRANCH` using the value provided via `--branch`, following the same mechanism as other `ConfigArgParse`-controlled options.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.