Tests which use deployers do not work on Helix
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 290
Description
The current design of test deployers requires the source code for the test projects to exist on the machine running tests _AND_ for the code to be buildable at test-time. This creates two problems:
1. In distributed testing on Helix, the source code is not present, only test binaries. Tests which rely on deployers are currently disabled
2. Simply copying the source code into a distributed test environment is insufficient because the projects are not currently written to build in a clean, vanilla "dotnet build" context.
### Potential solutions
Two options:
1. build test assets along with the test project and copy the test assets _as binaries_ into the test output. Deployers run on binaries (`dotnet foo.dll`) instead of on source (`dotnet publish && dotnet run`)
2. change the test asset code to be buildable in a 'vanilla' build context - don't build locally, only during test time. Deployers work the same, but test code should not build in VS or during build.cmd with the rest of the repo -- the code only builds during test execution.
Option 1 is preferable because it is easier to ensure the code can compile when we change APIs, and in many cases we're not especially interested in testing the behavior of the SDK.
Option 2 has some marginal benefit in that it reduces the up-front build time of the repo and makes the test payload smaller.
cc @Tratcher @anurse
Contributor guide
Assessment
This issue has not been assessed yet.