microsoft / microsoft/aspire

Foundry Local integration fails with current CLI and remote hosts

Open
#19,213 1 comment 0 reactions 0 assignees View on GitHub
area-app-model area-integrations triage:bot-seen
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

Issues #10937 and #12750 each cover part of this problem. This issue consolidates the currently broken Foundry Local scenarios addressed by PR #19197.

### Describe the bug

The current `RunAsFoundryLocal()` implementation is coupled to the legacy Foundry CLI `service` command and its foreground-process behavior. Current Foundry Local releases use a daemonized `server` command, so several supported or expected workflows do not work:

1. **Managed Foundry Local with the current CLI**
- Foundry CLI 0.10.1 exposes `foundry server`, not `foundry service`.
- Aspire invokes the legacy command and cannot reliably start, observe, or stop the current daemon.

2. **Daemonized server lifecycle**
- The modern start command exits after launching `foundrylocald`.
- Aspire currently treats the launching process as the service lifetime.
- The daemon inherits redirected output handles, which can also leave startup waiting indefinitely for EOF.

3. **Already-cached models**
- Aspire always enters the download path instead of loading an existing cached model first.
- A deployment can remain displayed as `Downloading` even though the model is already present locally.

4. **WSL2 or Linux AppHost connecting to Foundry Local on Windows**
- There is no API for supplying an existing Foundry Local endpoint.
- Aspire therefore attempts to manage Foundry Local on the AppHost machine instead of connecting to the reachable Windows-hosted service.

5. **Externally managed Foundry Local servers**
- There is no attach-only mode.
- Aspire cannot model an existing server without also assuming ownership of its startup, model downloads, model loading, and shutdown.

6. **Modern service and model health checks**
- Readiness relies on the tracked CLI process and `foundry service ps` rather than the server HTTP APIs.
- The integration does not use `/v1/models` for service health or `/models/loaded` for loaded-model readiness, with the legacy endpoint fallback.

7. **Exact remote model identifiers**
- Remote services can expose a concrete model ID that differs from the configured alias.
- There is no `LocalModelId` setting to ensure the connection properties and health checks use the actual loaded ID.

Azure-hosted Microsoft Foundry is unaffected. Applications embedding the Foundry Local native SDK directly are also unaffected because they do not use `RunAsFoundryLocal()`. Older CLI releases that still expose `foundry service` can continue to work locally.

### Expected Behavior

- Aspire-managed mode should support both legacy `foundry service` and current `foundry server` CLI generations.
- Cached models should be loaded before attempting another download.
- Service and model health should use the Foundry Local HTTP APIs.
- Users should be able to supply an existing HTTP/HTTPS endpoint that Aspire observes without owning its lifecycle or models.
- Remote deployments should allow configuring the exact model ID exposed by the existing service.
- AppHost shutdown should not fail when a best-effort CLI stop command reports an expected failure.

### Steps To Reproduce

#### Current CLI lifecycle

1. Install Foundry CLI 0.10.1.
2. Configure an AppHost:

```csharp
var chat = builder.AddFoundry("foundry")
.RunAsFoundryLocal()
.AddDeployment("chat", "phi-4-mini", "1", "Microsoft");
```

3. Start the AppHost.
4. Observe that Aspire depends on the removed `service` command and does not correctly manage the daemonized `server` lifecycle.

#### Cached model

1. Download `phi-4-mini` before starting the AppHost.
2. Start the AppHost with the deployment above.
3. Observe the deployment entering or remaining in the download state instead of loading the cached model directly.

#### WSL2 or external endpoint

1. Start Foundry Local on Windows and load a model.
2. Run the Aspire AppHost from WSL2 or another Linux environment.
3. Attempt to represent the existing Windows endpoint with `RunAsFoundryLocal()`.
4. Observe that there is no endpoint parameter or attach-only mode.

### Exceptions (if any)

Depending on the installed CLI version, the Foundry resource can fail to start because `foundry service` is unavailable. The dashboard can also remain in a `Downloading` state for an already-cached model.

### Aspire doctor output

N/A. The failures are specific to the Foundry Local hosting integration and CLI generation/lifecycle behavior.

### Anything else?

Related work and context:

- Fix: PR #19197
- Remote endpoint scenario: #10937
- Cached model/download state: #12750
- Upstream design context: https://github.com/luisquintanilla/foundry-local-aichatweb/issues/1
- Foundry Local describes the CLI and local server as optional because native SDK consumers can run inference in-process. Aspire's current managed hosting integration specifically uses the CLI/server workflow, so it still needs compatible lifecycle handling.

Contributor guide

Open the contributing guide

Research direction

Start with the RunAsFoundryLocal() integration and review PR #19197, along with issues #10937 and #12750. Trace managed and attach-only lifecycle behavior, cached-model handling, HTTP health checks, and remote model identifiers. Done means the current and legacy CLI workflows, cached models, external endpoints, and remote identifiers behave as described without breaking Azure-hosted Foundry.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.