Azure / Azure/AzureLocal-Supportability
Azure CLI fails on Azure Local nodes with "exit status 1" when run using 32-bit cmd.exe
- Dominant language
- PowerShell
- Stars
- 78
- Forks
- 60
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 5
Description
Originally reported here: https://github.com/Azure/azure-sdk-for-go/issues/25722
**Bug description**
We have a CLI extension `arcappliance` which calls into a Go binary for some commands. We started using `AzureCLICredential` from `azure-sdk-for-go` in the Go binary to perform ARM operations. `AzureCLICredential` internals calls Azure CLI via `cmd.exe` to get the token. This works in most cases, but we started seeing issues in Azure Local node environments.
This is the error we would see:
```
AzureCLICredential: exit status 1
```
Note: Azure Local uses a 32-bit build of Azure CLI. This is discussed below.
**Repro steps**
1. Create the following Go program in `cli_cred_repro.go`:
cli_cred_repro.go
```go
package main
import (
"context"
"fmt"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/cloud"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
)
func main() {
creds, err := azidentity.NewAzureCLICredential(nil)
if err != nil {
fmt.Printf("Failed to create credential: %v\n", err)
return
}
fmt.Println("Successfully created credential")
// Uncomment to apply the workaround
// err = os.Setenv("PATH", "C:\\Windows\\Sysnative;"+os.Getenv("PATH"))
// if err != nil {
// fmt.Printf("ERROR: failed to set PATH environment variable: %s\n", err.Error())
// return
// }
token, err := creds.GetToken(context.Background(), policy.TokenRequestOptions{
Scopes: []string{cloud.AzurePublic.Services[cloud.ResourceManager].Audience + "/.default"},
})
if err != nil {
fmt.Printf("Failed to get token: %v\n", err)
return
}
fmt.Printf("Successfully retrieved token with expiry: %s\n", token.ExpiresOn.String())
}
```
2. Compile both Windows 64 bit and 32 bit binaries:
```
$ GOOS=windows GOARCH=amd64 go build -o cli_cred_repro.exe cli_cred_repro.go
$ GOOS=windows GOARCH=386 go build -o cli_cred_repro-32bit.exe cli_cred_repro.go
```
3. Copy the binaries to an Azure Local node and execute them. The 32-bit version will fail until you uncomment the workaround code:
```
[v-host1]: PS C:\> .\cli_cred_repro.exe
Successfully created credential
Successfully retrieved token with expiry: 2025-12-05 01:34:41 +0000 UTC
[v-host1]: PS C:\> .\cli_cred_repro-32bit.exe
Successfully created credential
Failed to get token: AzureCLICredential: exit status 1
```
**Expected behavior**
The 32-bit program should get a token from `AzureCLICredential` successfully, which requires running Azure CLI in a 32-bit `cmd.exe` successfully.
**Environment (please complete the following information):**
- Build [release version, e.g. 10.2405.0.24]: 12.2601.1002.32 (not build specific)
- One-node or multi-node: The issue happens on individual nodes
- Production or non-production: All
- Region [e.g. East US]: All
**Cause**
Azure Local currently uses a 32-bit build of Azure CLI, even though it runs under 64-bit Windows. I found that the 32-bit `cmd.exe` (located at `C:\Windows\SysWOW64\cmd.exe`) on Azure Local nodes is unable to run any scripts, including Azure CLI.
With a script at `C:\test.cmd` that only contains `@echo "test"`, I observed that the 64-bit `cmd.exe` could execute it but not the 32-bit `cmd.exe`:
```
[v-host1]: PS C:\> cmd.exe /c "C:\test.cmd"
"test"
[v-host1]: PS C:\> C:\Windows\SysWOW64\cmd.exe /c "C:\test.cmd"
[v-host1]: PS C:\> $LASTEXITCODE
1
```
Since `az` on Windows runs a script called `az.cmd`, this means Azure CLI will fail to run under the 32-bit `cmd.exe`. I don't know why this is the behavior, but it explains why `AzureCLICredential` fails with no output other than `exit status 1`.
On my local machine I am able to run the script successfully with the 32-bit `cmd.exe`, so it is unclear how Azure Local is different.
**Workaround**
I'm not sure what the long-term fix is (likely it is in Azure Local), but there is a work around.
The Go program can prepend `C:\Windows\Sysnative;` to `PATH` before `AzureCLICredential` is used. This will cause `cmd.exe` to resolve to the 64-bit version instead of 32-bit.
```go
if runtime.GOOS == "windows" && runtime.GOARCH == "386" {
err := os.Setenv("PATH", "C:\\Windows\\Sysnative;"+os.Getenv("PATH"))
if err != nil {
return err
}
}
```
**Screenshots**
N/A
**Correlation ID**
[Collect logs](https://learn.microsoft.com/azure-stack/hci/manage/get-support-for-deployment-issues#perform-standalone-log-collection) and share the correlation ID here.
N/A
**Additional context**
See above "Cause" and "Workaround" sections.
Contributor guide
Assessment
This issue has not been assessed yet.