Azure / Azure/azure-functions-host

Flex Consumption: DrainMode cycling on alwaysReady HTTP instance without any deployment or config change

Open
#11,632 1 comment 0 reactions 0 assignees View on GitHub
area: flex-consumption bug Needs: Triage (Functions)
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

## Description

A Flex Consumption function app with `alwaysReady: [{ name: "http", instanceCount: 1 }]` and `siteUpdateStrategy: "Recreate"` repeatedly enters DrainMode without any deployment, configuration change, or scaling trigger. This causes the single always-ready instance to go down, dropping incoming HTTP webhooks during the drain/restart window.

Setting `alwaysReady` to 2 instances resolves the issue (one instance survives while the other drains), but this shouldn't be necessary when no site update is occurring.

## Reproduction

1. Create a Flex Consumption function app (Python 3.13, 2048 MB instance)
2. Configure `alwaysReady: [{ name: "http", instanceCount: 1 }]`
3. Configure `siteUpdateStrategy: { type: "Recreate" }`
4. Deploy the app once and wait for it to stabilize
5. Send HTTP requests (Zoom webhooks in our case) over the next 1-2 hours without making any changes
6. Observe DrainMode events in App Insights with no corresponding deployment or config change

## Observed behavior

- DrainMode is triggered repeatedly (~every 30-60 minutes) with no deployment or configuration change
- The single always-ready instance is drained and restarted, causing a cold start
- HTTP requests arriving during the drain window are dropped (Zoom webhooks return errors, never reach application code)
- App Insights logs show the pattern:
```
DrainMode mode enabled
Calling StopAsync on the registered listeners
Stopped the listener ... for function 'zoom_webhook'
[gap — instance down]
Host started (Nms)
Job host started
```
- We observed 5+ cold starts in a 2-hour window with zero deployments

## Expected behavior

With `alwaysReady: 1` and no site update occurring, the instance should remain stable and not enter DrainMode. DrainMode should only be triggered by actual site updates (deployments, config changes) as documented in [Site update strategies in Flex Consumption](https://learn.microsoft.com/en-us/azure/azure-functions/flex-consumption-site-updates).

## Workaround

Setting `alwaysReady` to 2 HTTP instances ensures one instance survives DrainMode cycling. All requests are handled without drops. However, this doubles the always-ready billing cost for what should be a single-instance workload.

## Environment

- **Function App**: Flex Consumption plan, East US 2
- **Runtime**: Python 3.13
- **Instance size**: 2048 MB
- **siteUpdateStrategy**: Recreate
- **Functions**: 4 (HTTP webhook receiver, OAuth callback, Graph webhook, timer-triggered subscription manager)
- **Traffic**: Low volume (~10-50 HTTP requests/hour)

## Related issues

- #10527 — Flex Consumption timer functions killed by unexpected DrainMode
- Azure/azure-webjobs-sdk#2782 — DrainMode canceling in-flight invocations

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Flex Consumption setup described for Python 3.13 with one always-ready HTTP instance, then inspect the App Insights DrainMode events and their timing against deployments or configuration changes. Compare behavior with two always-ready instances and verify that an instance does not enter DrainMode without a site update and that HTTP requests are not dropped during the test window.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp, python
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.