Azure / Azure/Azure-Functions

High CPU utilization in Azure Durable Function on AKS due to excessive file watcher threads (600–800 threads per pod)

Open
#2,646 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
PowerShell
Stars
1.1k
Forks
215
Avg merge
4h 2m
Merged PRs (30d)
1

Description

## Description

We are running an **Azure Durable Function App** on **AKS with KEDA** using a **custom Docker image** based on the official Azure Functions runtime. We are consistently observing **high CPU utilization** caused by a very large number of **file watcher threads (600–800)** being spawned inside the function pod.

This occurs intermittently:

* Sometimes in **one pod**
* Sometimes across **two pods**
* Even when traffic is low or idle

Despite explicitly disabling file watching via environment variables and `host.json`, the issue persists.

---

## Environment

* **Platform**: Azure Kubernetes Service (AKS)
* **Scaling**: KEDA
* **Function type**: Durable Functions
* **Runtime**: Azure Functions v4
* **.NET version**: .NET 8
* **OS**: Linux container

---

## Dockerfile

```dockerfile
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS installer-env

FROM mcr.microsoft.com/azure-functions/dotnet:4-dotnet8.0

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true \
DOTNET_HOSTBUILDER__RELOADCONFIGONCHANGE=false \
DOTNET_USE_POLLING_FILE_WATCHER=1 \
AzureFunctionsJobHost__fileWatchingEnabled=false \
WEBSITE_RUN_FROM_PACKAGE=1 \
ASPNETCORE_URLS=http://+:8080 \
WEBSITE_HOSTNAME=localhost:8080
```

---

## host.json

```json
{
"version": "2.0",
"logging": {
"fileLoggingMode": "never",
"console": {
"isEnabled": "false"
},
"logLevel": {
"default": "Information",
"Host": "Error"
},
"applicationInsights": {
"samplingSettings": {
"isEnabled": false,
"maxTelemetryItemsPerSecond": 500
}
}
},
"extensions": {
"durableTask": {
"storageProvider": {
"controlQueueBatchSize": 32,
"controlQueueBufferThreshold": 256,
"controlQueueVisibilityTimeout": "00:05:00",
"maxQueuePollingInterval": "00:00:05",
"partitionCount": 4,
"workItemQueueVisibilityTimeout": "00:05:00",
"useAppLease": true
},
"maxConcurrentActivityFunctions": 40,
"maxConcurrentOrchestratorFunctions": 40
}
},
"watchFiles": false,
"watchDirectories": false,
"fileWatchingEnabled": false
}
```

---

## Observed Behavior

* CPU spikes to high levels even during low or no load
* Thread dump shows **600–800 file watcher–related threads**
* High CPU correlates directly with these file watcher threads
* Scaling events (scale down / scale up via KEDA) seem to increase the likelihood
* Issue reproduces even with:

* `watchFiles=false`
* `fileWatchingEnabled=false`
* `DOTNET_HOSTBUILDER__RELOADCONFIGONCHANGE=false`
* `WEBSITE_RUN_FROM_PACKAGE=1`

---

## Expected Behavior

* File watching should be **fully disabled** when explicitly configured
* Durable Function pods should not spawn hundreds of file watcher threads
* CPU utilization should remain low during idle or low-traffic periods

---

## Additional Notes

* This setup is fully **containerized** (not App Service)
* Running Azure Functions **on AKS with KEDA**
* Looks like file watching might still be enabled internally or re-enabled during scaling events
* This causes unnecessary CPU consumption and impacts cluster cost and stability

---

## Request

* Clarification on **why file watcher threads are still being created**
* Guidance on **fully disabling file watching** in Azure Functions running on AKS
* Confirmation if this is a **known issue** with:

* Azure Functions v4
* Durable Functions
* .NET 8
* KEDA-based scaling

---

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.