dotnet / dotnet/project-system

Migrate integration tests to internal repository and build/run process

Open
#8,358 2 comments 0 reactions 0 assignees View on GitHub
Area-Infrastructure Area-Testing Triage-Approved
Dominant language
C#
Stars
1k
Forks
415
PR merge metrics
No merged PRs in 30d

Description

Currently, integration tests do not function. The legacy MyGet package source is no longer referenced by the repo, and the tooling required for APEX (very old version) was being pulled from that source. All the [APEX tooling](https://devdiv.visualstudio.com/DevDiv/_wiki/wikis/DevDiv.wiki/2287/Apex) (new versions) are published internally. A majority of integration tests exist within the VS repo itself, but as we all know, using that repo is not a fun endeavor. Additionally, integration tests run on an unreleased version of VS (in-progress development versions), so there is no avenue where it makes sense to have integration tests to be ran/exposed publicly.

My proposal is as such:
- Create an internal repo that only contains our integration tests
- This keeps us out of the VS repo (🥳)
- Integration tests need the internal APEX packages to be built/developed
- Create a pipeline for building/publishing the integration tests
- Integration tests should also be accessible to OptProf, which requires them to be published to a certain location
- Create a pipeline for running the integration tests
- Grabs the latest published version of the tests (as opposed to building them itself)
- Grabs the target project-system PR/version (can pull from the build produced by another pipeline)
- Runs the tests
- Determine how results are provided (maybe a bot that produces comment on the GitHub PR?)
- Determine when/how integration tests are run
- Per PR? Schedule? During Integration? Some of these depend on how long it takes to run the tests.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.