[CI][Test] The behavior of incremental test is not correct sometimes
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
Issue:
When the diff between the PR branch (source) and master (extension repo)/dev (cli repo) branch (target) contains not only the PR changes, the incremental test will test the current PR changes as well other changes and it's not a correct behavior as incremental test should only test the changes in the current PR. This will cause some PRs fail to pass the CI.
Example: https://github.com/Azure/azure-cli-extensions/pull/1892
Potential solution:
CI rebases the PR branch on latest master/dev branch, then diff between the PR branch and master/dev branch, then test based on the diff.
Currently we leverage ADO diff functionality, if it doesn't support the solution above, we need use git commands ourselves.
Current workaround:
PR owner rebases the PR branch on latest master/dev branch manually.
Contributor guide
Assessment
This issue has not been assessed yet.