.WithAdminDeploymentScriptStorage with more than one AzureSQL DB reference causes DeploymentScriptContainerGroupNotFound error
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 201
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Describe the bug
It appears there's a race condition of sorts when you use .WithAdminDeploymentScriptStorage and wire up more than one resource with a reference to a SQL DB, the deployment script runs and cleans up from the first resource before the second can process, which results in:
1m 13s ✗ provision-migrations-roles-azuresql │ ╶─────╴ │ — Failed to provision migrations-roles-azuresql:
Deployment failed: Error code = DeploymentScriptContainerGroupNotFound, Message = The deployment script was not able to find container group
'd5kurzv6c56w4azscripts', please make sure that another process or tool did not remove it. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more
deployment script information.
1m 45s ✗ provision-api-roles-azuresql │ ╶───────╴│ — Failed to provision api-roles-azuresql:
Deployment failed: Error code = DeploymentScriptContainerGroupNotFound, Message = The deployment script was not able to find container group
'd5kurzv6c56w4azscripts', please make sure that another process or tool did not remove it. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more
deployment script information.
I tried to solve this with a suggested "renamer" approach to name the separate scripts differently, but either I didn't wire that up correctly, or there's another underlying issue. It's intermittent because it's a race condition, so sometimes it'll fail, but if I just re-run the pipeline, but 99% of the time it'll work fine on the subsequent run.
Below is the code I tried to add to get around this issue but it doesn't seem to be fully resolving the issue.
```csharp
builder.Services.TryAddEventingSubscriber();
private sealed class SqlDeploymentScriptRenamer : IDistributedApplicationEventingSubscriber
{
public Task SubscribeAsync(
IDistributedApplicationEventing eventing,
DistributedApplicationExecutionContext executionContext,
CancellationToken cancellationToken)
{
eventing.Subscribe((e, _) =>
{
foreach (var rolesResource in e.Model.Resources.OfType()
.Where(r => r.Name.EndsWith("-roles-azuresql", StringComparison.Ordinal)))
{
new FakeBuilder(rolesResource).ConfigureInfrastructure(infra =>
{
var script = infra.GetProvisionableResources().OfType().Single();
script.Name = BicepFunction.Take(
BicepFunction.Interpolate($"script-{BicepFunction.GetUniqueString(rolesResource.GetBicepIdentifier(), BicepFunction.GetDeployment().Name)}"), BicepScriptNameMaxLength);
});
}
return Task.CompletedTask;
});
return Task.CompletedTask;
}
}
private sealed class FakeBuilder(T resource) : IResourceBuilder
where T : IResource
{
public T Resource => resource;
public IDistributedApplicationBuilder ApplicationBuilder => throw new NotSupportedException();
public IResourceBuilder WithAnnotation(
TAnnotation annotation,
ResourceAnnotationMutationBehavior behavior = ResourceAnnotationMutationBehavior.Append)
where TAnnotation : IResourceAnnotation
{
throw new NotSupportedException();
}
}
```
### Expected Behavior
I expect the deployment to be successful every time.
### Steps To Reproduce
1. Setup an Azure SQL Server and DB resource.
2. Create something like a simple API
3. Create another resource. In my case it's a MigrationService, a worker project that just runs migrations in the cloud after it's deployed
4. Give both projects a reference to the DB
5. Deploy - after a few deployments you're likely to run into this race condition.
### Exceptions (if any)
_No response_
### Aspire doctor output
.NET SDK
✅ .NET 10.0.108 installed (x64)
Container Runtime
✅ Docker v29.4.3: running (auto-detected (default)) ← active
Environment
✅ HTTPS development certificate is trusted
⚠️ HTTPS development certificate has an older version (v5)
Run 'aspire certs clean' to remove all certificates, then run 'aspire certs trust' to create and trust a new one.
See: https://aka.ms/aspire-prerequisites#dev-certs
Details:
Older certificate versions (< v4) may not support all certificate trust scenarios.
Summary: 3 passed, 1 warnings, 0 failed
For detailed prerequisites: https://aka.ms/aspire-prerequisites
### Anything else?
_No response_
Contributor guide
Research direction
Start with WithAdminDeploymentScriptStorage and the AzureProvisioningResource infrastructure for multiple Azure SQL references, focusing on AzurePowerShellScript naming and deployment-script cleanup. Reproduce the API and migration-service setup repeatedly, then verify that separate resources no longer remove or conflict over the same container group and deployments succeed consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp, sql
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100