Azure / Azure/azure-cli-extensions
[acrcssc] Cannot respond to request for authentication during OCI artifact push - root cause identified, not Cloud Shell-specific
- Dominant language
- Python
- Stars
- 454
- Forks
- 1.7k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 64
Description
### Related issues
- Azure/azure-cli-extensions#10003
- Azure/azure-cli#32817
Those reports attribute this to Azure Cloud Shell's credential handoff. I hit the identical failure on a plain local Linux install (non-Cloud-Shell) and traced it to a concrete bug in how `acrcssc` uses `oras-py`'s `TokenAuth` backend — it does not appear to be environment-specific.
### Environment
```
azure-cli: 2.75.0 (also reproduced on 2.90.0 before downgrading for an unrelated SDK issue)
acrcssc: 1.0.0b7
oras (vendored): 0.2.25
```
### Repro
```bash
az acr supply-chain workflow create \
-r -g -t continuouspatchv1 \
--config ./continuouspatching.json --schedule 1d --run-immediately
```
Fails with:
```
Failed to push OCI artifact to ACR: Cannot respond to request for authentication.
```
`--dry-run` succeeds (it never pushes), confirming the bug is isolated to the OCI artifact push step.
### Root cause
`_ociartifactoperations.py::_get_acr_token()` shells out to `az acr login --expose-token`, which returns an ACR **refresh token** (explicitly *not* a usable access token — `az` itself warns about this). `_oras_client()` then does:
```python
token = _get_acr_token(registry.name, subscription)
client = OrasClient(hostname=str.lower(registry.login_server), auth_backend="token")
client.login(BEARER_TOKEN_USERNAME, token)
```
But `oras.provider.Registry.login()` only ever calls `self.auth.set_basic_auth(username, password)` — it **never** calls `set_token_auth()`. So `TokenAuth.token` stays `None` after login.
Separately, `oras.provider.Registry.do_request()` contains:
```python
if headers is not None and isinstance(self.auth, oras.auth.TokenAuth):
headers.update(self.auth.get_auth_header())
```
`TokenAuth.get_auth_header()` unconditionally returns `{"Authorization": "Bearer %s" % self.token}`. Since `self.token is None`, this attaches a literal `Authorization: Bearer None` header on the **first** request, before any real 401 challenge/response cycle happens.
ACR sees a malformed-but-present Bearer credential and responds `403` **without** a `WWW-Authenticate` header (that's only sent on clean, unauthenticated 401s). `TokenAuth.authenticate_request()` then can't recover:
```python
authHeaderRaw = original.headers.get("Www-Authenticate")
if not authHeaderRaw:
logger.debug("Www-Authenticate not found in original response, cannot authenticate.")
return headers, False # -> do_request() raises ValueError("Cannot respond to request for authentication.")
```
I confirmed this by replaying the exact same "raw refresh token as Bearer" request manually — ACR returns `401 insufficient_scope` *with* `WWW-Authenticate` in isolation, but the premature/poisoned first request in the real flow behaves differently and the extension never gets a chance to recover.
I also confirmed the `oras-py` "basic" backend (`auth_backend="basic"`) is **not** a valid alternative — its `authenticate_request()` just resends raw HTTP Basic auth to the blob endpoint, which ACR's data-plane API doesn't accept; no real OAuth2 exchange happens there either.
### Fix
Rather than relying on `oras`'s broken token negotiation, perform the ACR `refresh_token` → scoped `access_token` exchange explicitly, and hand the resulting token directly to the auth backend via `set_token_auth()` (bypassing `login()`'s broken basic-auth-only path entirely):
**Before** (`azext_acrcssc/helper/_ociartifactoperations.py`):
```python
def _oras_client(registry):
resourceid = parse_resource_id(registry.id)
subscription = resourceid[SUBSCRIPTION]
try:
token = _get_acr_token(registry.name, subscription)
client = OrasClient(hostname=str.lower(registry.login_server), auth_backend="token")
client.login(BEARER_TOKEN_USERNAME, token)
logger.debug(f"Login to ACR {registry.name} completed successfully.")
except Exception as exception:
raise AzCLIError(f"Failed to login to Artifact Store ACR {registry.name}: {exception}")
return client
```
**After:**
```python
import requests
def _oras_client(registry):
resourceid = parse_resource_id(registry.id)
subscription = resourceid[SUBSCRIPTION]
try:
refresh_token = _get_acr_token(registry.name, subscription)
login_server = str.lower(registry.login_server)
# oras-py's TokenAuth backend does not perform the ACR
# refresh_token -> access_token exchange correctly (it sends a
# premature "Bearer None" header on the first request, and cannot
# recover afterwards). Perform the exchange explicitly instead.
scope = f"repository:{CSSC_WORKFLOW_POLICY_REPOSITORY}/{CONTINUOUSPATCH_OCI_ARTIFACT_CONFIG}:pull,push"
exchange_resp = requests.post(
f"https://{login_server}/oauth2/token",
data={
"grant_type": "refresh_token",
"service": login_server,
"scope": scope,
"refresh_token": refresh_token,
},
timeout=30,
)
exchange_resp.raise_for_status()
access_token = exchange_resp.json()["access_token"]
client = OrasClient(hostname=login_server, auth_backend="token")
client.auth.set_token_auth(access_token)
logger.debug(f"Login to ACR {registry.name} completed successfully.")
except Exception as exception:
raise AzCLIError(f"Failed to login to Artifact Store ACR {registry.name}: {exception}")
return client
```
I applied this locally against `~/.azure/cliextensions/acrcssc/` and confirmed `az acr supply-chain workflow create ... --run-immediately` now completes successfully end-to-end (artifact push, ARM task deployment, and workflow trigger all succeed).
### Suggested longer-term fix
Ideally this gets fixed upstream in `oras-py` itself (`TokenAuth`/`do_request` shouldn't attach a `Bearer None` header when no token has been obtained yet), but the workaround above avoids depending on a fix there and is a minimal, self-contained change to `acrcssc`.
Happy to open a PR with this change if useful.
Contributor guide
Research direction
Start in azext_acrcssc/helper/_ociartifactoperations.py, especially _get_acr_token() and _oras_client(), then reproduce with the supplied az acr supply-chain workflow create command and --run-immediately. Review the oras-py TokenAuth behavior and the ACR /oauth2/token exchange described in the issue. Done means the command completes the artifact push, ARM task deployment, and workflow trigger without the authentication error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100