Azure / Azure/azure-cli

az network application-gateway wait --custom with != JMESPath expression returns immediately instead of blocking

Open
#32,961 1 comment 0 reactions 2 assignees Claimed by @thomas-temby View on GitHub
act-quality-productivity-squad Auto-Assign Azure CLI Team customer-reported Network question
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

### Describe the bug

`az network application-gateway wait --custom "provisioningState!='Updating'"` returns immediately (~2 seconds) even while the gateway's `provisioningState` is `Updating`. The JMESPath `!=` expression does not evaluate correctly — it always passes, regardless of the actual state.

The built-in `--updated` flag works correctly and blocks for the full duration of the operation. This means `provisioningState` is accurate, but `--custom` with a `!=` expression fails to evaluate it properly.

**Side-by-side proof during the same gateway update operation:**

| Method | Gateway state | Behavior | Duration |
|---|---|---|---|
| `--custom "provisioningState!='Updating'"` | `Updating` | Returned immediately (broken) | ~2s |
| `--updated` | `Updating` | Blocked until operation completed (correct) | ~47s |
| `az appgw show --query provisioningState` | `Updating` | Correctly returned `"Updating"` | N/A |

**Impact:** This bug makes it impossible to use `--custom` as a pre-flight check to coordinate concurrent updates. We used this in CI/CD pipelines to avoid `CanceledAndSupersededDueToAnotherOperation` errors when multiple pipelines update different backend pools on the same gateway. Because `--custom` always returned immediately, every pipeline blew through the wait and collided.

### Related command

```
az network application-gateway wait --custom
```

### Errors

No error is raised — `--custom` silently returns a false positive. The downstream consequence is that a second pipeline submits while the first is still in progress, causing:

```
(CanceledAndSupersededDueToAnotherOperation) Operation PutApplicationGatewayOperation
was canceled and superseded by another PutApplicationGatewayOperation.
Code: CanceledAndSupersededDueToAnotherOperation
```

### Issue script & Debug output

**Reproduction requires two terminals and one App Gateway.**

Terminal 1 — trigger a real pool update (must make an actual change):

```bash
az network application-gateway address-pool update \
--gateway-name "agw-example" \
--resource-group "rg-example" \
--name "my-api-pool" \
--servers 10.0.1.1 10.0.1.2 10.0.1.3
```

Terminal 2 — while Terminal 1 is running (~10 seconds in), run the `--custom` wait:

```bash
# This should block but returns in ~2 seconds (BUG)
az network application-gateway wait \
--name "agw-example" \
--resource-group "rg-example" \
--custom "provisioningState!='Updating'" \
--interval 5 \
--timeout 120

# Immediately after, confirm the gateway is still Updating
az network application-gateway show \
--name "agw-example" \
--resource-group "rg-example" \
--query "provisioningState" -o tsv
# Returns: Updating
```

Terminal 3 — for comparison, run `--updated` at the same time:

```bash
# This correctly blocks for ~50 seconds
az network application-gateway wait \
--name "agw-example" \
--resource-group "rg-example" \
--updated
```

**Our test results (PowerShell, local machine polling every 2 seconds):**

We ran a monitoring script that polled `az appgw show --query provisioningState` every 2 seconds while triggering a pipeline that called `az address-pool update`. The `show` command correctly returned `Updating` for ~59 seconds during the operation.

We then ran a second test that detected `Updating` via the `show` poll, immediately launched `--custom "provisioningState!='Updating'"`, and it returned in ~2 seconds while `show` still reported `Updating`:

```
[16:48:16.121] *** UPDATING DETECTED (via show) ***
=== Test A: --custom "provisioningState!='Updating'" ===
[16:48:19.978] --custom returned after 2s
provisioningState right after --custom returned: Updating
*** BROKEN: --custom returned while gateway is still Updating! ***
```

In a separate test, `--updated` blocked correctly for ~47 seconds until the operation completed.

Debug output from the `--custom` call shows it performs a GET, receives the full gateway JSON, and exits with code 0 after a single poll — suggesting the JMESPath expression evaluates to a truthy value even when `provisioningState` equals `Updating`.

### Expected behavior

`az network application-gateway wait --custom "provisioningState!='Updating'"` should block while `provisioningState` is `Updating` and only return once the condition is actually true (i.e., when `provisioningState` changes to `Succeeded`).

The `!=` JMESPath expression should evaluate to `false` when the property equals the compared value, just as the `==` expression (used internally by `--updated`) correctly evaluates to `false` when the property does not match.

### Environment Summary

```
azure-cli 2.84.0 (mcr.microsoft.com/azure-cli:2.84.0, Linux)
azure-cli 2.70.0 (Windows, local testing)
```

Both versions exhibit the same behavior.

### Additional context

**Workaround:** Use `--updated` instead of `--custom "provisioningState!='Updating'"`. The `--updated` flag correctly checks `provisioningState=='Succeeded'` internally and blocks for the full duration of an in-progress operation.

This bug may not be specific to Application Gateway — any resource where `--custom` uses a `!=` JMESPath expression against `provisioningState` may be affected. We have not tested other resource types.

Related issues:
- #21156 — `CanceledAndSupersededDueToAnotherOperation` during concurrent App Gateway updates
- #27771 — Concurrent pool updates silently revert each other's changes

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.