Azure / Azure/azure-cli

[CI][Test] The behavior of incremental test is not correct sometimes

Open
#14,586 1 comment 0 reactions 1 assignee Claimed by @haroldrandom View on GitHub
act-platform-engineering-squad CI feature-request Test Framework
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

Issue:
When the diff between the PR branch (source) and master (extension repo)/dev (cli repo) branch (target) contains not only the PR changes, the incremental test will test the current PR changes as well other changes and it's not a correct behavior as incremental test should only test the changes in the current PR. This will cause some PRs fail to pass the CI.

Example: https://github.com/Azure/azure-cli-extensions/pull/1892

Potential solution:
CI rebases the PR branch on latest master/dev branch, then diff between the PR branch and master/dev branch, then test based on the diff.
Currently we leverage ADO diff functionality, if it doesn't support the solution above, we need use git commands ourselves.

Current workaround:
PR owner rebases the PR branch on latest master/dev branch manually.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.