microsoft-foundry / microsoft-foundry/foundry-samples

"Failed to create Aml RP virtual workspace due to System.Exception: Failed async operation", "Received 404 from a service request". - 15b-private-network-standard-agent-setup-byovnet EXAMPLE ERROR

Open
#314 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Bicep
Stars
445
Forks
494
Avg merge
11h 35m
Merged PRs (30d)
38

Description

I am following the AI Foundry with VNET Injection directions and have successfully deployed my Terraform configuration into EastUS region. When I attempt to deploy into WestUS region I am receiving "Failed to create Aml RP virtual workspace due to System.Exception: Failed async operation", "Received 404 from a service request".

My working VNET in EASTUS is 172.18.0.0/22 with AgentHost subnet of 172.18.1.0/24 with Microsoft.Apps/environments service delegation;
Not working VNET in WESTUS is 172.18.24.0/22 with AgentHost subnet of 172.18.25.0/24 with Microsoft.Apps/environments service delegation;

Both VNET's use the same private DNS listener target for DNS resolution and have the same private DNS zone's linked, so DNS resolution is identical for both. I have triple checked everything matches up... resource providers, DNS, code, etc.. the only difference is the subscription and region. I have been spinning wheels here for several weeks and Microsoft Support is still working with me, but they appear stumped themselves currently. Has anyone that experienced this issue ever actually found a reason for it? Please do not link me to #309 as that is not a solution to this issue.

I am following private VNET injection example 15b and this is the resource creation that is failing; Full error is at the bottom.
(https://github.com/azure-ai-foundry/foundry-samples/blob/main/samples/microsoft/infrastructure-setup-terraform/15b-private-network-standard-agent-setup-byovnet/code/main.tf)

Create the AI Foundry resource

resource "azapi_resource" "ai_foundry" {
provider = azapi.workload_subscription

type = "Microsoft.CognitiveServices/accounts@2025-06-01"
name = "aifoundry${random_string.unique.result}"
parent_id = "/subscriptions/${var.subscription_id_resources}/resourceGroups/${var.resource_group_name_resources}"
location = var.location
schema_validation_enabled = false

body = {
kind = "AIServices",
sku = {
name = "S0"
}
identity = {
type = "SystemAssigned"
}

properties = {
  # Support both Entra ID and API Key authentication for underlining Cognitive Services account
  disableLocalAuth = false

  # Specifies that this is an AI Foundry resource
  allowProjectManagement = true

  # Set custom subdomain name for DNS names created for this Foundry resource
  customSubDomainName = "aifoundry${random_string.unique.result}"

  # Network-related controls
  # Disable public access but allow Trusted Azure Services exception
  publicNetworkAccess = "Disabled"
  networkAcls = {
    defaultAction = "Allow"
  }

  # Enable VNet injection for Standard Agents
  networkInjections = [
    {
      scenario                   = "agent"
      subnetArmId                = var.subnet_id_agent
      useMicrosoftManagedNetwork = false
    }
  ]
}

}
}

Error: Failed to create/update resource
with azapi_resource.exampleaifoundryname
on westus-azureaifoundry.tf line 128, in resource "azapi_resource" "exampleaifoundryname-001":
resource "azapi_resource" "exampleaifoundryname-001" {
creating/updating Resource: (ResourceId "/subscriptions//resourceGroups/rg-exampleaifoundryname-westus/providers/Microsoft.CognitiveServices/accounts/exampleaifoundryname-westus" / Api Version "2025-06-01"): GET https://management.azure.com/subscriptions//providers/Microsoft.CognitiveServices/locations/West US/operationResults/

RESPONSE 200: 200 OK
ERROR CODE: OperationError

{
"id": "/subscriptions//providers/Microsoft.CognitiveServices/locations/West US/operationResults/",
"name": "",
"status": "Failed",
"startTime": "2025-09-29",
"endTime": "2025-09-29",
"error": {
"code": "OperationError",
"message": "Failed to Create the resource. Provisioning state: Failed",
"details": [
{
"code": "ResourceProviderError",
"message": "Failed to create Aml RP virtual workspace due to System.Exception: Failed async operation {\n "status": "Failed",\n "error": {\n "code": "InternalServerError",\n "message": "Received 404 from a service request"\n }\n}\n at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AsyncOperationHelper.PollUntilCompleteAsync[T](TraceContext traceContext, HttpResponseMessage response, HttpClient httpClient, Int32 timeoutInMin, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AsyncOperationHelper.cs:line 117\n at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AmlRpClient.PutAmlRpVirtualHubAsync(TraceContext traceContext, Dictionary2 headers, ResourceIdentityInArm resourceIdentity, String primaryUserAssignedIdentity, String internalId, KeyVaultProperties keyVaultProperties, String subscriptionId, String resourceGroupName, String workspaceName, List1 networkInjections, PublicNetworkAccessType publicNetworkAccessType, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AmlRpClient.cs:line 119\n at Microsoft.CognitiveServices.ResourceProvider.AmlRp.AmlRpService.PutAmlRpWorkspaceAsync(TraceContext traceContext, ResourceRequestContext requestContext, WorkerMessageRequestContext workerMessageRequestContext, IResourceEntity insertedResourceEntity, CancellationToken cancellationToken) in /__w/1/s/src/Common/ResourceProvider/AmlRp/AmlRpService.cs:line 124\n at Microsoft.CognitiveServices.ResourceProvider.Worker.MessageProcessor.ProcessInternalAsync(TraceContext traceContext, RpWorkerQueueMessage message, CancellationToken cancellationToken) in /__w/1/s/src/ResourceProvider/Rp.WorkerRole/AccountProvisioning/MessageProcessor.cs:line 223"
}
]
}
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with samples/microsoft/infrastructure-setup-terraform/15b-private-network-standard-agent-setup-byovnet/code/main.tf and compare the working EastUS and failing WestUS deployments, including the AI Foundry resource request and AgentHost subnet settings. Reproduce the WestUS creation failure and trace the operation result for the 404 service request. Done means the regional cause is identified and a verified configuration or documented service limitation explains the failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, terraform
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.