Azure / Azure/azure-functions-host
Adaptive Parallelism
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 38
Description
#### I would like better (non-static) control over the number of concurrent messages within my app
I have an app with many functions (30 or so), each watching a single queue. Normally these queues all receive moderate levels of work, so the levels of parallelism are set to newBatchThreshold 16, batchSize:8. This means that when the server gets busy, it doesn't overload too bad (cpu does go to 100% but no further issues), the queues just backlog.
However, if just one of the queues gets a very large set of messages, and none of the other queues have anything to do, i can end up with an app service with only 10% cpu usage, and no way to get the remaining 90% of the cpu focused on the one busy queue.
If i turn newBatchThreshold up to 200, then 1 queue can scale to consume 100% of the cpu available, at the cost of the server stability if i have messages across all 30 queues.
#### I would like a metric based batch trigger
In the prototypical image rescaling example for azure functions no specific batch threshold could be set, as you could do many small images in parallel easier than a few big images.
An alternative could be "optimize for app service host cpu" -- new batches are allowed to be pulled whenever cpu is below x%.
If you synchronously wait on database requests, then this can also be used to auto-scale against an unknown database load.
#### Describe alternatives you've considered
The proposed solution relies on cpu metrics which may not be appropriate in various hosting scenarios.
an alternate solution is a new batch threshold in host.json:
// The threshold beyond which new batches of messages are Not pulled, even if the queue specific "newBatchThreshold" has not been reached.
"serverBatchThreshold": 1024
// The threshold at which a new batch of messages will be fetched.
// ... New batches of messages will be pulled until the number of messages being processed is
// greater than this threshold,
// **or the number of messages across all workers is greater than the serverBatchThreshold**
"newBatchThreshold": 16
This solution doesn't take into account that some messages are harder to process than others, and/or require more/less waiting on external IO.
#### Additional context
Consumption plans would have to solve this problem separatly to Hosted App services.
I suspect that people on consumption plans would like a maximum total parallelism level request.
Contributor guide
Research direction
No source file or test is named. Start by tracing how host.json configures newBatchThreshold and how queue workers track concurrent messages; compare the proposed CPU-based and serverBatchThreshold approaches, including consumption plans. Done requires an agreed adaptive-parallelism design with coverage for the stated hosting scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100