Foundry Local integration fails with current CLI and remote hosts
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
Issues #10937 and #12750 each cover part of this problem. This issue consolidates the currently broken Foundry Local scenarios addressed by PR #19197.
### Describe the bug
The current `RunAsFoundryLocal()` implementation is coupled to the legacy Foundry CLI `service` command and its foreground-process behavior. Current Foundry Local releases use a daemonized `server` command, so several supported or expected workflows do not work:
1. **Managed Foundry Local with the current CLI**
- Foundry CLI 0.10.1 exposes `foundry server`, not `foundry service`.
- Aspire invokes the legacy command and cannot reliably start, observe, or stop the current daemon.
2. **Daemonized server lifecycle**
- The modern start command exits after launching `foundrylocald`.
- Aspire currently treats the launching process as the service lifetime.
- The daemon inherits redirected output handles, which can also leave startup waiting indefinitely for EOF.
3. **Already-cached models**
- Aspire always enters the download path instead of loading an existing cached model first.
- A deployment can remain displayed as `Downloading` even though the model is already present locally.
4. **WSL2 or Linux AppHost connecting to Foundry Local on Windows**
- There is no API for supplying an existing Foundry Local endpoint.
- Aspire therefore attempts to manage Foundry Local on the AppHost machine instead of connecting to the reachable Windows-hosted service.
5. **Externally managed Foundry Local servers**
- There is no attach-only mode.
- Aspire cannot model an existing server without also assuming ownership of its startup, model downloads, model loading, and shutdown.
6. **Modern service and model health checks**
- Readiness relies on the tracked CLI process and `foundry service ps` rather than the server HTTP APIs.
- The integration does not use `/v1/models` for service health or `/models/loaded` for loaded-model readiness, with the legacy endpoint fallback.
7. **Exact remote model identifiers**
- Remote services can expose a concrete model ID that differs from the configured alias.
- There is no `LocalModelId` setting to ensure the connection properties and health checks use the actual loaded ID.
Azure-hosted Microsoft Foundry is unaffected. Applications embedding the Foundry Local native SDK directly are also unaffected because they do not use `RunAsFoundryLocal()`. Older CLI releases that still expose `foundry service` can continue to work locally.
### Expected Behavior
- Aspire-managed mode should support both legacy `foundry service` and current `foundry server` CLI generations.
- Cached models should be loaded before attempting another download.
- Service and model health should use the Foundry Local HTTP APIs.
- Users should be able to supply an existing HTTP/HTTPS endpoint that Aspire observes without owning its lifecycle or models.
- Remote deployments should allow configuring the exact model ID exposed by the existing service.
- AppHost shutdown should not fail when a best-effort CLI stop command reports an expected failure.
### Steps To Reproduce
#### Current CLI lifecycle
1. Install Foundry CLI 0.10.1.
2. Configure an AppHost:
```csharp
var chat = builder.AddFoundry("foundry")
.RunAsFoundryLocal()
.AddDeployment("chat", "phi-4-mini", "1", "Microsoft");
```
3. Start the AppHost.
4. Observe that Aspire depends on the removed `service` command and does not correctly manage the daemonized `server` lifecycle.
#### Cached model
1. Download `phi-4-mini` before starting the AppHost.
2. Start the AppHost with the deployment above.
3. Observe the deployment entering or remaining in the download state instead of loading the cached model directly.
#### WSL2 or external endpoint
1. Start Foundry Local on Windows and load a model.
2. Run the Aspire AppHost from WSL2 or another Linux environment.
3. Attempt to represent the existing Windows endpoint with `RunAsFoundryLocal()`.
4. Observe that there is no endpoint parameter or attach-only mode.
### Exceptions (if any)
Depending on the installed CLI version, the Foundry resource can fail to start because `foundry service` is unavailable. The dashboard can also remain in a `Downloading` state for an already-cached model.
### Aspire doctor output
N/A. The failures are specific to the Foundry Local hosting integration and CLI generation/lifecycle behavior.
### Anything else?
Related work and context:
- Fix: PR #19197
- Remote endpoint scenario: #10937
- Cached model/download state: #12750
- Upstream design context: https://github.com/luisquintanilla/foundry-local-aichatweb/issues/1
- Foundry Local describes the CLI and local server as optional because native SDK consumers can run inference in-process. Aspire's current managed hosting integration specifically uses the CLI/server workflow, so it still needs compatible lifecycle handling.
Contributor guide
Research direction
Start with the RunAsFoundryLocal() integration and review PR #19197, along with issues #10937 and #12750. Trace managed and attach-only lifecycle behavior, cached-model handling, HTTP health checks, and remote model identifiers. Done means the current and legacy CLI workflows, cached models, external endpoints, and remote identifiers behave as described without breaking Azure-hosted Foundry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, devtools
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100