microsoft / microsoft/aspire

CLI E2E tests that use bare bash (non-Docker) fail with "aspire: command not found" outside PR context

Open
#15,930 0 comments 0 reactions 1 assignee Claimed by @radical View on GitHub
area-cli disabled-tests testing ☑️
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Description

Several CLI E2E tests in `Aspire.Cli.EndToEnd.Tests` are **structurally broken** when run outside of a PR context — specifically in the quarantine (`tests-quarantine.yml`) and outerloop (`tests-outerloop.yml`) scheduled workflows. These tests fail immediately with `bash: aspire: command not found` because no CLI binary is available on PATH.

## Root Cause

The non-Docker CLI E2E tests guard their CLI installation behind `IsRunningInCI`, which checks for `GITHUB_PR_NUMBER` and `GITHUB_PR_HEAD_SHA` environment variables:

```csharp
// CliE2ETestHelpers.cs
internal static bool IsRunningInCI =>
!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_NUMBER")) &&
!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_HEAD_SHA"));
```

When `isCI` is `false` (as in scheduled runs with no PR context), the CLI setup block is skipped entirely — but the test still proceeds to call `aspire new`, which fails:

```csharp
if (isCI)
{
await auto.InstallAspireCliFromPullRequestAsync(prNumber, counter);
await auto.SourceAspireCliEnvironmentAsync(counter);
await auto.VerifyAspireCliVersionAsync(commitSha, counter);
}
// No else — no fallback to GA release or any other install method
await auto.AspireNewAsync(ProjectName, counter, useRedisCache: false); // 💥
```

**Docker-based tests don't have this problem** because `DetectDockerInstallMode()` falls back to `DockerInstallMode.GaRelease`, which downloads the GA release via `get-aspire-cli.sh` inside the container.

## Affected Tests

These quarantined tests consistently fail with `aspire: command not found` in scheduled runs:

| Test | Quarantine Issue |
|------|------------------|
| `KubernetesPublishTests.CreateAndPublishToKubernetes` | #15870 |
| `DockerDeploymentTests.CreateAndDeployToDockerCompose` | #15882 |
| `DockerDeploymentTests.CreateAndDeployToDockerComposeInteractive` | #15871 |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.