microsoft / microsoft/aspire

Allow limiting parallelism of aspire deploy/publish pipeline steps

Open
#18,714 1 comment 0 reactions 0 assignees View on GitHub
area-deployment
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

### Is your feature request related to a problem? Please describe.

`aspire deploy` (and `aspire publish`) execute all independent pipeline steps fully in parallel with no concurrency cap. For an AppHost with several compute resources requiring an image build, this means every `build-*` and `push-*` step for every resource starts at the same time — in our case 8 services, each with multiple image layers, pushed simultaneously to the same private registry.

Against a registry sitting behind a reverse proxy (Traefik, in our case) this routinely causes push failures under load:

```
error from registry: invalid content range
unexpected status from PUT request to https:///v2/.../blobs/uploads/...: 499 status code 499
failed to do request: Head "https:///v2/.../blobs/...": net/http: TLS handshake timeout
```

Retrying `aspire deploy` a few times eventually gets everything through (each retry a few more images already have their layers on the registry), but a single deploy essentially never succeeds end-to-end when every image push starts at once. Tuning proxy/registry timeouts helps, but a client-side way to bound how many pushes run concurrently would let us avoid overloading a registry that reasonably can't take 7-8 large concurrent uploads.

### Describe the solution you'd like

A supported way to cap how many independent pipeline steps run concurrently during `aspire deploy`/`aspire publish` — e.g.:

- A CLI flag, such as `aspire deploy --max-parallel-steps `, or
- A `PipelineOptions` setting configurable from the AppHost (e.g. `builder.Services.Configure(o => o.MaxDegreeOfParallelism = 2)`), or
- A per-resource/step opt-in to a named concurrency group (e.g. all `push-*` steps sharing a "registry-push" group capped at N concurrent).

Any of these would let us throttle registry pushes specifically without serializing the entire pipeline (build steps that don't hit the network could stay fully parallel).

### Additional context

I looked at the pipeline executor source (`src/Aspire.Hosting/Pipelines/DistributedApplicationPipeline.cs`) to see if this was already possible. `ExecuteAsync` always calls `ExecuteStepsAsTaskDag`, which starts every step with an unthrottled `Task.Run` — no semaphore, no `MaxDegreeOfParallelism`, nothing configurable that I could find. There's also an `ExecuteStepsSequentially` / `ExecuteStepSequentiallyAsync(stepName, ...)` path, but it's only used internally for running a single step (and its dependencies) via `aspire do ` — it isn't reachable for a full deploy/publish run.

As a workaround today, we pre-warm the registry by pushing each image individually with `docker push` before running `aspire deploy`, so most layers already exist by the time the deploy step re-pushes them — but this doesn't help on a first-time deploy of new images (e.g. after adding a new service to the AppHost).

### Environment

- Aspire CLI: 13.4.6+87fe259e4fc244c599019a7b1304c85a1488f248 (build 13.400.626.31906)
- AppHost: C# AppHost, `AddContainerRegistry` + `WithContainerRegistry` (`ASPIRECOMPUTE003` preview) targeting a private Docker registry
- OS: Windows 11
- .NET SDK: 10.0

Contributor guide

Open the contributing guide

Research direction

Start in src/Aspire.Hosting/Pipelines/DistributedApplicationPipeline.cs, reading ExecuteAsync and ExecuteStepsAsTaskDag to understand how deploy and publish launch independent steps. Compare that path with ExecuteStepsSequentially and ExecuteStepSequentiallyAsync; done should provide a supported concurrency cap for full deploy/publish runs without requiring all steps to execute serially.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, docker
Domain
devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.