Aspire do support for run mode related artifacts
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Is your feature request related to a problem? Please describe the problem.
We run the apphost in CI using `aspire start` and then `aspire wait` on the critical resources.
But we kept on having timing issues due to network pulls of the different images from registry.
So we created a pipeline step to pre-pull all the runtime images.
However, the resource model in pipeline steps is totally different than in run mode, causing it to be harder to author, but not only that, when we will start to publish using `aspire publish` all the local emulators and prepackaged services (like redis) will be replaced with terraform and helm charts and not containers.
So we will lose the ability to use the apphost to tell us what the actual list of images needed to be pulled is.
### Describe the solution you'd like
I want to be able to register pipeline steps that execute with the `aspire run` context - access to the actual AppModel we use during run mode, and not publish mode.
### Additional context
Here is the snippet we had to implement and see how ackward it is to figure out what is an image and what is an executable and should not be pulled:
```csharp
// Prevents network saturation and Docker Hub / registry rate-limit errors.
var pullSemaphore = new SemaphoreSlim(maxConcurrentPulls, maxConcurrentPulls);
builder.Pipeline.AddStep(
name: "prepull-images",
action: ctx =>
{
ctx.Logger.LogInformation("prepull-images: all image pulls completed.");
return Task.CompletedTask;
});
builder.Pipeline.AddPipelineConfiguration(async configCtx =>
{
var existingStepNames = new HashSet(
configCtx.Steps.Select(s => s.Name),
StringComparer.OrdinalIgnoreCase);
var seenImages = new HashSet(StringComparer.OrdinalIgnoreCase);
foreach (var resource in configCtx.Model.Resources)
{
if (!resource.TryGetLastAnnotation(out var img))
{
continue;
}
if (resource.HasAnnotationOfType())
{
continue;
}
var hasRegistry = !string.IsNullOrWhiteSpace(img.Registry);
var registry = hasRegistry ? img.Registry!.TrimEnd('/') + "/" : "";
var tag = string.IsNullOrWhiteSpace(img.Tag) ? "latest" : img.Tag;
var imageRef = $"{registry}{img.Image}:{tag}";
var stepName = $"pull-{resource.Name}";
// Skip if already registered (guard against multiple invocations).
if (existingStepNames.Contains(stepName) || !seenImages.Add(imageRef))
{
continue;
}
string? platform = null;
foreach (var annotation in resource.Annotations.OfType())
{
var callbackCtx = new ContainerRuntimeArgsCallbackContext([], CancellationToken.None);
await annotation.Callback(callbackCtx);
var args = callbackCtx.Args.Select(a => a.ToString() ?? "").ToList();
var platformIdx = args.IndexOf("--platform");
if (platformIdx >= 0 && platformIdx + 1 < args.Count)
{
platform = args[platformIdx + 1];
break;
}
}
var platformCapture = platform;
builder.Pipeline.AddStep(
name: stepName,
action: async ctx =>
{
var platformArg = platformCapture is not null ? $"--platform {platformCapture} " : "";
// Throttle concurrent pulls to avoid network saturation and registry rate-limits.
await pullSemaphore.WaitAsync(ctx.CancellationToken);
try
{
ctx.Logger.LogInformation("Pulling {Image}...", imageRef);
var sw = Stopwatch.StartNew();
var psi = new ProcessStartInfo("docker", $"pull --quiet {platformArg}{imageRef}")
{
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
};
using var proc = Process.Start(psi)
?? throw new InvalidOperationException($"Failed to start 'docker pull' for {imageRef}");
await proc.WaitForExitAsync(ctx.CancellationToken);
sw.Stop();
if (proc.ExitCode != 0)
{
var stderr = await proc.StandardError.ReadToEndAsync(ctx.CancellationToken);
throw new InvalidOperationException(
$"docker pull failed for {imageRef} (exit {proc.ExitCode}): {stderr.Trim()}");
}
ctx.Logger.LogInformation("✓ {Image} pulled in {Elapsed:F1}s", imageRef, sw.Elapsed.TotalSeconds);
}
finally
{
pullSemaphore.Release();
}
},
requiredBy: new[] { "prepull-images" });
}
});
```
Contributor guide
Research direction
Start by tracing how `aspire run` builds the AppModel and how `AddPipelineConfiguration` currently receives the publish-mode resource model. Compare the run and publish contexts, including `ContainerImageAnnotation` and runtime arguments. Done means pipeline steps can access the run-mode AppModel and identify the runtime images needed for pre-pulling without relying on publish-mode resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, docker
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100