Azure / Azure/azure-cli-extensions

[containerapp] `az containerapp job start --image` replaces the requested image with the quickstart image when `--registry-identity system` is used

Open
#9,947 2 comments 1 reaction 3 assignees Claimed by @zhoxing-ms View on GitHub
Auto-Assign ContainerApp Service Attention Subscription
Dominant language
Python
Stars
454
Forks
1.7k
Avg merge
2d 19h
Merged PRs (30d)
64

Description

Source: https://github.com/Azure/azure-cli/issues/33548 (by @eric15342335)
Affected extension: `containerapp` (`src/containerapp/`)

---

<<>>
# Issue #33548 (by @eric15342335)
## Title
`az containerapp job start --image` replaces the requested image with the quickstart image when `--registry-identity system` is used

## Body
When starting an existing Azure Container Apps job with both:

''' powershell
--image
--registry-identity system
'''

the execution does not use the image supplied through `--image`.

Instead, Azure CLI submits the following public quickstart image in the execution template:

''' powershell
mcr.microsoft.com/k8se/quickstart:latest
'''

The existing job is already configured with:

* A system-assigned managed identity
* A private Azure Container Registry
* The registry identity set to `system`
* The requested private image in the stored job template

Therefore, no identity bootstrap should be required when starting an execution.

### Related command

[az containerapp job start](https://learn.microsoft.com/en-us/cli/azure/containerapp/job?view=azure-cli-latest#az-containerapp-job-start)

### Errors

No CLI exception or non-zero exit code is returned.

`az containerapp job start` completes successfully with exit code `0` and creates an execution. However, inspecting the resulting execution shows that the image supplied through `--image` was replaced.

The execution was started using:

''' powershell
$execution = az containerapp job start `
--resource-group "" `
--name "" `
--image ".azurecr.io/:" `
--container-name "" `
--cpu 0.25 `
--memory 0.5Gi `
--registry-identity system `
--query name `
--output tsv
'''

The resulting execution was inspected using:

''' powershell
az containerapp job execution show `
--resource-group "" `
--name "" `
--job-execution-name $execution `
--query 'properties.template.containers[].{name:name,image:image,cpu:resources.cpu,memory:resources.memory}' `
--output jsonc
'''

The inspection command returned:

''' json
[
{
"cpu": 0.25,
"image": "mcr.microsoft.com/k8se/quickstart:latest",
"memory": "0.5Gi",
"name": ""
}
]
'''

The command therefore reports success, but the execution uses `mcr.microsoft.com/k8se/quickstart:latest` instead of the private image explicitly supplied through `--image`.

### Issue script & Debug output

''' powershell
$ErrorActionPreference = "Stop"

$SubscriptionId = ""
$ResourceGroup = ""
$EnvironmentName = ""
$AcrName = ""
$WorkloadProfileName = "Consumption"

$Suffix = Get-Date -Format "MMddHHmmss"
$JobName = "cli-job-start-repro-$Suffix"
$ContainerName = "repro"
$Repository = "cli-job-start-repro"
$Tag = $Suffix
$SourceImage = "mcr.microsoft.com/k8se/quickstart-jobs:latest"

$ExecutionName = $null
$JobCreated = $false
$ImageImported = $false

function Assert-AzSuccess {
param([string]$Operation)

if ($LASTEXITCODE -ne 0) {
throw "$Operation failed with exit code $LASTEXITCODE."
}
}

try {
Write-Host "Selecting the test subscription"

az account set `
--subscription $SubscriptionId

Assert-AzSuccess "Selecting the subscription"

$LoginServerOutput = az acr show `
--name $AcrName `
--query loginServer `
--output tsv

Assert-AzSuccess "Reading the ACR login server"

$LoginServer = ($LoginServerOutput | Out-String).Trim()
$PrivateImage = "${LoginServer}/${Repository}:${Tag}"

Write-Host "Importing a small test image into the private ACR"

az acr import `
--name $AcrName `
--source $SourceImage `
--image "${Repository}:${Tag}" `
--force `
--output none

Assert-AzSuccess "Importing the test image"
$ImageImported = $true

Write-Host "Creating a temporary job on the Consumption workload profile"

az containerapp job create `
--resource-group $ResourceGroup `
--name $JobName `
--environment $EnvironmentName `
--trigger-type Manual `
--replica-timeout 300 `
--replica-retry-limit 0 `
--replica-completion-count 1 `
--parallelism 1 `
--image $PrivateImage `
--container-name $ContainerName `
--cpu 0.25 `
--memory 0.5Gi `
--workload-profile-name $WorkloadProfileName `
--mi-system-assigned `
--registry-server $LoginServer `
--registry-identity system `
--output none

Assert-AzSuccess "Creating the temporary job"
$JobCreated = $true

Write-Host ""
Write-Host "Stored job configuration"
Write-Host ""

az containerapp job show `
--resource-group $ResourceGroup `
--name $JobName `
--query '{identityType:identity.type,registries:properties.configuration.registries,workloadProfile:properties.workloadProfileName,containers:properties.template.containers[].{name:name,image:image,cpu:resources.cpu,memory:resources.memory}}' `
--output jsonc `
--debug

Assert-AzSuccess "Reading the stored job configuration"

Write-Host ""
Write-Host "Starting the execution with --image and --registry-identity system"
Write-Host ""

$ExecutionOutput = az containerapp job start `
--resource-group $ResourceGroup `
--name $JobName `
--image $PrivateImage `
--container-name $ContainerName `
--cpu 0.25 `
--memory 0.5Gi `
--registry-identity system `
--query name `
--output tsv `
--debug

Assert-AzSuccess "Starting the job execution"

$ExecutionName = ($ExecutionOutput | Out-String).Trim()

Write-Host ""
Write-Host "Execution name: $ExecutionName"
Write-Host ""
Write-Host "Resulting execution template"
Write-Host ""

az containerapp job execution show `
--resource-group $ResourceGroup `
--name $JobName `
--job-execution-name $ExecutionName `
--query 'properties.template.containers[].{name:name,image:image,cpu:resources.cpu,memory:resources.memory}' `
--output jsonc `
--debug

Assert-AzSuccess "Reading the execution template"
}
finally {
if ($ExecutionName) {
Write-Host ""
Write-Host "Stopping the temporary execution"

az containerapp job stop `
--resource-group $ResourceGroup `
--name $JobName `
--job-execution-name $ExecutionName `
--only-show-errors `
--output none
}

if ($JobCreated) {
Write-Host "Deleting the temporary job"

az containerapp job delete `
--resource-group $ResourceGroup `
--name $JobName `
--yes `
--only-show-errors `
--output none
}

if ($ImageImported) {
Write-Host "Deleting the temporary ACR image"

az acr repository delete `
--name $AcrName `
--image "${Repository}:${Tag}" `
--yes `
--only-show-errors `
--output none
}
}
'''

The complete reproduction script above was run with Azure CLI `2.87.0` and the `containerapp` extension `1.3.0b4`.

The existing job was successfully created with the requested private image:

''' json
{
"containers": [
{
"cpu": 0.25,
"image": ".azurecr.io/cli-job-start-repro:",
"memory": "0.5Gi",
"name": "repro"
}
],
"identityType": "SystemAssigned",
"registries": [
{
"identity": "system",
"server": ".azurecr.io"
}
],
"workloadProfile": "Consumption"
}
'''

The following command was then run:

''' powershell
$ExecutionOutput = az containerapp job start `
--resource-group "" `
--name "" `
--image ".azurecr.io/cli-job-start-repro:" `
--container-name "repro" `
--cpu 0.25 `
--memory 0.5Gi `
--registry-identity system `
--query name `
--output tsv `
--debug
'''

Relevant sanitized debug output:

''' json
CommandName: containerapp job start
ParameterSetName: --resource-group --name --image --container-name --cpu --memory --registry-identity --query --output --debug

Request URL:
https://management.azure.com/subscriptions//resourceGroups/
... [truncated, original was 11917 chars]

## Comments
### Comment by @yonzhan

Thank you for opening this issue, we will look into it.
<<>>

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.