CLI E2E tests that use bare bash (non-Docker) fail with "aspire: command not found" outside PR context
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
## Description
Several CLI E2E tests in `Aspire.Cli.EndToEnd.Tests` are **structurally broken** when run outside of a PR context — specifically in the quarantine (`tests-quarantine.yml`) and outerloop (`tests-outerloop.yml`) scheduled workflows. These tests fail immediately with `bash: aspire: command not found` because no CLI binary is available on PATH.
## Root Cause
The non-Docker CLI E2E tests guard their CLI installation behind `IsRunningInCI`, which checks for `GITHUB_PR_NUMBER` and `GITHUB_PR_HEAD_SHA` environment variables:
```csharp
// CliE2ETestHelpers.cs
internal static bool IsRunningInCI =>
!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_NUMBER")) &&
!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_HEAD_SHA"));
```
When `isCI` is `false` (as in scheduled runs with no PR context), the CLI setup block is skipped entirely — but the test still proceeds to call `aspire new`, which fails:
```csharp
if (isCI)
{
await auto.InstallAspireCliFromPullRequestAsync(prNumber, counter);
await auto.SourceAspireCliEnvironmentAsync(counter);
await auto.VerifyAspireCliVersionAsync(commitSha, counter);
}
// No else — no fallback to GA release or any other install method
await auto.AspireNewAsync(ProjectName, counter, useRedisCache: false); // 💥
```
**Docker-based tests don't have this problem** because `DetectDockerInstallMode()` falls back to `DockerInstallMode.GaRelease`, which downloads the GA release via `get-aspire-cli.sh` inside the container.
## Affected Tests
These quarantined tests consistently fail with `aspire: command not found` in scheduled runs:
| Test | Quarantine Issue |
|------|------------------|
| `KubernetesPublishTests.CreateAndPublishToKubernetes` | #15870 |
| `DockerDeploymentTests.CreateAndDeployToDockerCompose` | #15882 |
| `DockerDeploymentTests.CreateAndDeployToDockerComposeInteractive` | #15871 |
Contributor guide
Assessment
This issue has not been assessed yet.