Azure / Azure/azure-functions-host

Adaptive Parallelism

Open
#5,609 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

#### I would like better (non-static) control over the number of concurrent messages within my app
I have an app with many functions (30 or so), each watching a single queue. Normally these queues all receive moderate levels of work, so the levels of parallelism are set to newBatchThreshold 16, batchSize:8. This means that when the server gets busy, it doesn't overload too bad (cpu does go to 100% but no further issues), the queues just backlog.

However, if just one of the queues gets a very large set of messages, and none of the other queues have anything to do, i can end up with an app service with only 10% cpu usage, and no way to get the remaining 90% of the cpu focused on the one busy queue.

If i turn newBatchThreshold up to 200, then 1 queue can scale to consume 100% of the cpu available, at the cost of the server stability if i have messages across all 30 queues.

#### I would like a metric based batch trigger
In the prototypical image rescaling example for azure functions no specific batch threshold could be set, as you could do many small images in parallel easier than a few big images.

An alternative could be "optimize for app service host cpu" -- new batches are allowed to be pulled whenever cpu is below x%.

If you synchronously wait on database requests, then this can also be used to auto-scale against an unknown database load.

#### Describe alternatives you've considered

The proposed solution relies on cpu metrics which may not be appropriate in various hosting scenarios.

an alternate solution is a new batch threshold in host.json:
// The threshold beyond which new batches of messages are Not pulled, even if the queue specific "newBatchThreshold" has not been reached.
"serverBatchThreshold": 1024
// The threshold at which a new batch of messages will be fetched.
// ... New batches of messages will be pulled until the number of messages being processed is
// greater than this threshold,
// **or the number of messages across all workers is greater than the serverBatchThreshold**
"newBatchThreshold": 16

This solution doesn't take into account that some messages are harder to process than others, and/or require more/less waiting on external IO.

#### Additional context
Consumption plans would have to solve this problem separatly to Hosted App services.
I suspect that people on consumption plans would like a maximum total parallelism level request.

Contributor guide

Open the contributing guide

Research direction

No source file or test is named. Start by tracing how host.json configures newBatchThreshold and how queue workers track concurrent messages; compare the proposed CPU-based and serverBatchThreshold approaches, including consumption plans. Done requires an agreed adaptive-parallelism design with coverage for the stated hosting scenarios.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.