Azure / Azure/azure-functions-host

Dynamically adjust consumption scale

Open
#4,443 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

#### What problem would the feature you're requesting solve? Please describe.
When targeting an endpoint that is not infinitely scalable, it is difficult to determine the correct scale values to use and ideal scale values may change over time.

Consider these scenarios:

1. Assume you are hitting a service endpoint and are assuming you have XX% of the service's resources to work with. You set the scale settings appropriately and move on. Later, someone performs a large job that eats at your assumed resources and the Function app starts experiencing a high number of failures. It keeps trying and failing, and, eventually, the host gives up and lots of triggers never get processed.
2. Assume two Functions in a single Function app that are hitting the same endpoint. Traffic is low so the scale setting is working fine. Traffic starts increasing and the Function app starts experiencing a high number of failures. It keeps trying and failing, and, eventually, the host gives up and lots of triggers never get processed.
3. Assume a Function app's scale settings were set by assuming a particular tier at the endpoint service. Traffic starts to increase and the business can now justify a higher tier endpoint service. The endpoint service is scaled up, but the Function app can't dynamically scale with it. The Function app ends up with a giant backlog, because the Function app isn't working fast enough now. Once this is diagnosed, the consumption app can't be scaled up as simply as the endpoint service because a new commit to source control and a new deployment will be necessary to adjust the Function app settings.

#### Describe the solution you'd like
Function app can return a TimeoutException that will cause the host to scale back the number of instances running.

#### Describe alternatives you've considered
1. Host detects that function instances are taking longer than usual and scales down.
2. Function app sets a timeout value for each function. If the timeout value is exceeded, host should scale down the number of instances. Maybe
3. Triggers never stop occurring, even after X errors (maybe only for certain exception types).
4. The Function app is in control of the trigger retry strategy, allowing exponential or random retry periods.

#### Additional context
This is related to https://github.com/Azure/azure-functions-host/issues/4301.

Contributor guide

Open the contributing guide

Research direction

Start by reading the related issue #4301 and compare its context with the three scaling scenarios and proposed alternatives here. This issue needs an agreed design and acceptance criteria for how the host should react to timeouts, failures, or changing endpoint capacity before implementation can be considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.