Azure / Azure/azure-functions-host
Host reports 403 AuthorizationPermissionMismatch on Blobs with correct RBAC, indexes no functions, and reports nothing to App Insights
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
### Investigative information
- **Timestamp**: 2026-08-21, ~09:20–13:55 UTC
- **Function App version**: host `4.1052.300.0`, `FUNCTIONS_EXTENSION_VERSION=~4`
- **Region**: West Europe
- **Runtime**: `NODE|24`, Linux, `kind: functionapp,linux`
- **Plan**: Basic B1 (dedicated), `alwaysOn: true`
- **Storage connection**: identity-based, **system-assigned** managed identity
### Summary
A Function App host reaches `state: Error` with a single 403 `AuthorizationPermissionMismatch`
against Blob Storage, and therefore **indexes no functions at all** — `az functionapp function list`
returns empty — while the site reports `Running` and the package is deployed on disk.
The identity holds `Storage Blob Data Owner`, `Storage Queue Data Contributor` and `Reader` on the
target storage account, with **no ABAC conditions** and **no deny assignments**. The host's own
internal containers (`azure-webjobs-hosts`, `azure-webjobs-secrets`) **do exist** in that account,
which means blob access worked at least once.
**The same ARM template, same runtime, same role set, deployed to a different subscription, works
correctly and indexes its function.**
### Repro steps
1. Deploy, from one ARM/Bicep template, a Linux Function App on a **dedicated B1** plan alongside a
storage account, with `AzureWebJobsStorage` configured for identity:
- `AzureWebJobsStorage__accountName`
- `AzureWebJobsStorage__blobServiceUri`
- `AzureWebJobsStorage__queueServiceUri`
2. In the same deployment, create role assignments for the app's **system-assigned** identity on the
storage account: `Storage Blob Data Owner`, `Storage Queue Data Contributor`, `Reader`.
3. Deploy code with `az functionapp deployment source config-zip` (a pre-built bundle; no remote build).
4. Restart the app and query `GET /admin/host/status`.
### Expected behavior
The host starts and indexes the functions declared by the deployed package (a single blob-triggered
function, Node v4 programming model).
### Actual behavior
```
state: Error
version: 4.1052.300.0
errors: 1
Azure.Storage.Blobs: This request is not authorized to perform this operation using this permission.
RequestId: 535edfec-801e-00d8-7269-310f07000000
Time: 2026-08-21T12:32:06.7484680Z
Status: 403 (This request is not authorized to perform this operation using this permission.)
ErrorCode: AuthorizationPermissionMismatch
```
Further RequestIds with the same error, in case they help correlate server side:
`6d57152f-901e-00eb-2f61-3150ac000000` (11:36:11Z), `bc5daf1a-801e-0015-3879-2bcc07000000`.
Container log shows a repeating cycle:
```
Creating container with image: azure-functions/noble/linuxdedicated:4.1052.300-0-node24-appservice
Container is running.
Site startup probe succeeded after 65.5 seconds.
Site is running with deployment version:
Site started.
Container is terminating. Grace period: 5 seconds. <-- ~21 s after "Site started"
Container is terminated.
Site: stopped.
```
### What the error does **not** tell us
The message names `Azure.Storage.Blobs` but not the operation or the URI, so there is no way from the
outside to know which permission is missing. That is the core of this report: with `Storage Blob Data
Owner` + `Reader` granted, no conditions and no deny assignments, **which blob operation is being
rejected?**
### Ruled out, with evidence
| Hypothesis | How it was ruled out |
|---|---|
| Missing data-plane roles | all three present, listed by `--scope` with their assignment IDs |
| ABAC conditions on the assignments | `condition` is `null` on all four assignments on the account |
| Deny assignments | `GET .../providers/Microsoft.Authorization/denyAssignments` on the account returns `[]` |
| Creation path (per #10970) | `azure-webjobs-hosts` and `azure-webjobs-secrets` **exist** — so container creation succeeded |
| Storage account configuration | **identical across 11 fields** to the working environment: `networkRuleSet.defaultAction: Allow`, `bypass: None`, no IP/VNet rules, `allowSharedKeyAccess`, `publicNetworkAccess`, `defaultToOAuthAuthentication`, `minimumTlsVersion`, `allowBlobPublicAccess`, `allowCrossTenantReplication`, `kind`, `sku` |
| VNet integration | none configured; no VNets in the resource group |
| Package not deployed | SCM deployment history: status 4 (Success) ×3; rsync of 1,244 files / 12.45 MB reported "Completed successfully" |
| Missing app settings | full app-setting name set identical to the working environment |
| Unsupported runtime | the working environment runs the same `NODE|24` and indexes fine |
| Contention on the shared plan | the co-located web app was deployed and stabilised; symptom unchanged |
| Deployment order | reproduced with the code deployed as the very last operation |
| Application Insights as a diagnostic | **unusable here**: the same query returns zero rows in the *working* environment too, so its emptiness proves nothing. `/admin/host/status` was the only source that reported anything |
### Two observations that may matter
**1. `Get Blob Service Properties` is not covered by the data roles.** We add `Reader` deliberately,
because that service-level read is required at startup and `Storage Blob Data Owner` does not grant
`Microsoft.Storage/storageAccounts/blobServices/read`. The documented guidance suggests `Storage
Account Contributor`, which we avoid because it includes `listkeys/action` — the exact credential path
that identity-based connections exist to remove. If `Reader` is genuinely insufficient for some other
startup operation, it would help enormously to have that documented, or to have a narrower built-in
role for it.
**2. The failure is completely silent from a monitoring standpoint.** No telemetry, no failed
executions, `state: Running` on the site, and a green deployment. The only way we found to detect it
was polling `/admin/host/status` from a scheduled job. This is the same complaint as #10970, which was
closed — the observability gap it describes is still very much present in `4.1052.300.0`.
### Related
- #10970 — same class of silent failure. Our containers exist, so this is not the creation path, but the
observability problem described there is exactly what made this take a full day to characterise.
- #8189 — ARM template redeploys wiping deployment-tool state and making functions disappear. Different
mechanism (consumption + `WEBSITE_RUN_FROM_PACKAGE`), same family of problem.
Contributor guide
Research direction
Start with GET /admin/host/status and compare the failing and working subscriptions using the reported Azure.Storage.Blobs 403 RequestIds, identity roles, and startup configuration. Trace the host startup path that performs the rejected blob operation and verify the dedicated B1 deployment with the Node v4 package. Done means the host indexes the deployed function, identifies or documents the required permission, and exposes a useful failure signal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, node.js
- Domain
- authentication, backend, cloud, observability
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100