Azure / Azure/bicep-types-az

Azure Open AI deployment error. Unable to re-run the bicep script because of quota errors.

Open
#1,844 4 comments 0 reactions 0 assignees View on GitHub
Needs: Triage :mag: No RP Github RP: AzureOpenAI
Dominant language
TypeScript
Stars
108
Forks
44
Avg merge
18h 53m
Merged PRs (30d)
29

Description

**Bicep version**
Bicep CLI version 0.21.1 (d4acbd2a9f)

**Describe the bug**
If your rerun the provided **sample** script, it throws quota error even though there is **no** change in deployment script.

```
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity . This set TPM rate of deployment.
}
```
**Error**
```
Inner Errors:
{"code": "InsufficientQuota", "message": "The specified capacity '155' of account deployment is bigger than available capacity '10' for UsageName 'Tokens Per Minute (thousands) - GPT-35-Turbo'."}
```

**To Reproduce**
Below is sample script called **deploy.bicep** that you can use for local reproduction. Replace it with your AOAI instance name and region
```python
// Define an existing Cognitive Services resource using the 2023-05-01 API version
// https://learn.microsoft.com/en-us/azure/templates/microsoft.cognitiveservices/2023-05-01/accounts?pivots=deployment-language-bicep

// Declare the location parameter with a default value of 'canadaeast'
// This parameter specifies the location where the Azure OpenAI Cognitive Services resource will be created
@description('Specifies the location for resources.')
param location string = 'northcentralus'

// Declare the AOAIInstanceName parameter, which represents the name of the Azure OpenAI Cognitive Services resource
// This parameter allows you to set a custom name for the resource
@description('Name of Azure Open AI Cognitive Services resource')
param AOAIInstanceName string='mynorthcentralus123' // example

// Create the myAzureOpenAI resource, which represents the Azure OpenAI Cognitive Services account
// This resource creation sets the name, location, kind, SKU, and custom subdomain name properties
resource myAzureOpenAI 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
name: AOAIInstanceName // Set the resource name to the value of the AOAIInstanceName parameter
location: location // Set the resource location to the value of the location parameter
kind: 'OpenAI' // Set the resource kind to 'OpenAI'
sku: {
name: 'S0' // Set the SKU name to 'S0'
tier: 'Standard' // Set the SKU tier to 'Standard'
}
properties: {
customSubDomainName: AOAIInstanceName // Set the custom subdomain name to the value of the AOAIInstanceName parameter
publicNetworkAccess: 'Disabled'
networkAcls: { // Add this block to define network access control lists
defaultAction: 'Allow'
virtualNetworkRules: []
ipRules: []
}
}
}

// Define an array of Cognitive Services deployments with details
// This array contains a single deployment with model information, RAI policy name, version upgrade option, and SKU
var myAzureOpenAIDeployment = [
{
displayName: 'mync1' // Display name for the deployment
model: {
format: 'OpenAI' // Model format
name: 'gpt-35-turbo' // Model name
version: '0613' // Model version
}
raiPolicyName: 'Microsoft.Default' // RAI policy name for the deployment
versionUpgradeOption: 'OnceNewDefaultVersionAvailable' // Version upgrade option for the deployment
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity . This set TPM rate of deployment.
}
}
]

// Create a Model Deployment resource for each deployment in the myAzureOpenAIDeployment array
// Use batchSize(1) decorator to limit parallel deployments as they are not supported
@batchSize(1)
resource aoaiModelDeployment 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [for deployment in myAzureOpenAIDeployment: {
name: deployment.displayName // Set the deployment name to the display name from the array
parent: myAzureOpenAI // Set the parent resource of the deployment to the myAzureOpenAI resource
properties: {
model: deployment.model // Set the model properties of the deployment from the array
raiPolicyName: deployment.raiPolicyName // Set the RAI policy name of the deployment from the array
versionUpgradeOption: deployment.versionUpgradeOption // Set the version upgrade option of the deployment from the array
}
sku: deployment.sku // Set the SKU properties of the deployment from the array
}]

// Output the capacity of the first deployment in the aoaiModelDeployment array
// This output shows the capacity value of the model deployment created in the script
output Capacity int = aoaiModelDeployment[0].sku.capacity

```
Run this sample script
**az deployment group create --resource-group NorthCentralUS --template-file ./deploy.bicep**

**Expected Result :**
Deployment should succeed without any errors

**Actual Result :**
```
{"code": "InvalidTemplateDeployment", "message": "The template deployment 'deploy' is not valid according to the validation procedure. The tracking id is '57aa6077-48aa-44a8-8160-cb530bd9477b'. See inner errors for details."}

Inner Errors:
{"code": "InsufficientQuota", "message": "The specified capacity '155' of account deployment is bigger than available capacity '10' for UsageName 'Tokens Per Minute (thousands) - GPT-35-Turbo'."}
```

**Additional context**
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity. This set TPM rate of deployment.
}

Value in *capacity* is used to check for **available** quota without taking into account that it is an **existing** deployment and not a new capacity requirement.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the deploy.bicep reproduction and run it with `az deployment group create --resource-group NorthCentralUS --template-file ./deploy.bicep`. Investigate the Microsoft.CognitiveServices/accounts/deployments@2023-05-01 behavior when the deployment already exists and compare it with the reported InsufficientQuota response. Done means rerunning the unchanged sample succeeds without treating the existing capacity as a new quota requirement.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.