Azure Open AI deployment error. Unable to re-run the bicep script because of quota errors.
- Dominant language
- TypeScript
- Stars
- 108
- Forks
- 44
- Avg merge
- 18h 53m
- Merged PRs (30d)
- 29
Description
**Bicep version**
Bicep CLI version 0.21.1 (d4acbd2a9f)
**Describe the bug**
If your rerun the provided **sample** script, it throws quota error even though there is **no** change in deployment script.
```
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity . This set TPM rate of deployment.
}
```
**Error**
```
Inner Errors:
{"code": "InsufficientQuota", "message": "The specified capacity '155' of account deployment is bigger than available capacity '10' for UsageName 'Tokens Per Minute (thousands) - GPT-35-Turbo'."}
```
**To Reproduce**
Below is sample script called **deploy.bicep** that you can use for local reproduction. Replace it with your AOAI instance name and region
```python
// Define an existing Cognitive Services resource using the 2023-05-01 API version
// https://learn.microsoft.com/en-us/azure/templates/microsoft.cognitiveservices/2023-05-01/accounts?pivots=deployment-language-bicep
// Declare the location parameter with a default value of 'canadaeast'
// This parameter specifies the location where the Azure OpenAI Cognitive Services resource will be created
@description('Specifies the location for resources.')
param location string = 'northcentralus'
// Declare the AOAIInstanceName parameter, which represents the name of the Azure OpenAI Cognitive Services resource
// This parameter allows you to set a custom name for the resource
@description('Name of Azure Open AI Cognitive Services resource')
param AOAIInstanceName string='mynorthcentralus123' // example
// Create the myAzureOpenAI resource, which represents the Azure OpenAI Cognitive Services account
// This resource creation sets the name, location, kind, SKU, and custom subdomain name properties
resource myAzureOpenAI 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
name: AOAIInstanceName // Set the resource name to the value of the AOAIInstanceName parameter
location: location // Set the resource location to the value of the location parameter
kind: 'OpenAI' // Set the resource kind to 'OpenAI'
sku: {
name: 'S0' // Set the SKU name to 'S0'
tier: 'Standard' // Set the SKU tier to 'Standard'
}
properties: {
customSubDomainName: AOAIInstanceName // Set the custom subdomain name to the value of the AOAIInstanceName parameter
publicNetworkAccess: 'Disabled'
networkAcls: { // Add this block to define network access control lists
defaultAction: 'Allow'
virtualNetworkRules: []
ipRules: []
}
}
}
// Define an array of Cognitive Services deployments with details
// This array contains a single deployment with model information, RAI policy name, version upgrade option, and SKU
var myAzureOpenAIDeployment = [
{
displayName: 'mync1' // Display name for the deployment
model: {
format: 'OpenAI' // Model format
name: 'gpt-35-turbo' // Model name
version: '0613' // Model version
}
raiPolicyName: 'Microsoft.Default' // RAI policy name for the deployment
versionUpgradeOption: 'OnceNewDefaultVersionAvailable' // Version upgrade option for the deployment
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity . This set TPM rate of deployment.
}
}
]
// Create a Model Deployment resource for each deployment in the myAzureOpenAIDeployment array
// Use batchSize(1) decorator to limit parallel deployments as they are not supported
@batchSize(1)
resource aoaiModelDeployment 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [for deployment in myAzureOpenAIDeployment: {
name: deployment.displayName // Set the deployment name to the display name from the array
parent: myAzureOpenAI // Set the parent resource of the deployment to the myAzureOpenAI resource
properties: {
model: deployment.model // Set the model properties of the deployment from the array
raiPolicyName: deployment.raiPolicyName // Set the RAI policy name of the deployment from the array
versionUpgradeOption: deployment.versionUpgradeOption // Set the version upgrade option of the deployment from the array
}
sku: deployment.sku // Set the SKU properties of the deployment from the array
}]
// Output the capacity of the first deployment in the aoaiModelDeployment array
// This output shows the capacity value of the model deployment created in the script
output Capacity int = aoaiModelDeployment[0].sku.capacity
```
Run this sample script
**az deployment group create --resource-group NorthCentralUS --template-file ./deploy.bicep**
**Expected Result :**
Deployment should succeed without any errors
**Actual Result :**
```
{"code": "InvalidTemplateDeployment", "message": "The template deployment 'deploy' is not valid according to the validation procedure. The tracking id is '57aa6077-48aa-44a8-8160-cb530bd9477b'. See inner errors for details."}
Inner Errors:
{"code": "InsufficientQuota", "message": "The specified capacity '155' of account deployment is bigger than available capacity '10' for UsageName 'Tokens Per Minute (thousands) - GPT-35-Turbo'."}
```
**Additional context**
sku: {
name: 'Standard' // SKU name
capacity: 155 // SKU capacity. This set TPM rate of deployment.
}
Value in *capacity* is used to check for **available** quota without taking into account that it is an **existing** deployment and not a new capacity requirement.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the deploy.bicep reproduction and run it with `az deployment group create --resource-group NorthCentralUS --template-file ./deploy.bicep`. Investigate the Microsoft.CognitiveServices/accounts/deployments@2023-05-01 behavior when the deployment already exists and compare it with the reported InsufficientQuota response. Done means rerunning the unchanged sample succeeds without treating the existing capacity as a new quota requirement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100