Azure / Azure/Azure-Functions

Scaling behavior (EP plans) - low frequency input events, many (same) ActivityFunctions launched

Open
#1,834 9 comments 0 reactions 1 assignee Claimed by @balag0 View on GitHub
area: documentation
Dominant language
PowerShell
Stars
1.1k
Forks
215
Avg merge
4h 2m
Merged PRs (30d)
1

Description

Hi,

First, many thanks for Azure functions the ecosystem!

After several days of tests and research, we still struggle to adjust the scaling behavior to fit our needs.

- We run a ML inference algo in a "Python + durable functions + custom docker + ElasticPremium" context.
- We have a low frequency event input. Basically users upload a CSV file, and we get it using a BlobTrigger (we plan to use HTTPTriggers later).
This file can be quite long (up to 1000 lines), and each line is a computation.
- For each line, we launch a computation (In an activity function), and this computation is quite CPU intensive (>70% CPU) and takes 2 to 4 mins to complete (on EP4 plan)

Sometimes, we're OK with a computation 'batch' running several hours on 1 instance, but for some business needs, we would like to accelerate and benefit from the scaling behavior.

Last time, we had 1 'batch' of 250 computations to do quickly (1 csv file with 250 lines) :
- we maxed the plan to EP4 (4 cores)
- we set up the plan scale to "Always ready": 1, "Maximum burst" & "Maximum scale out limit" to 10 (see screenshot)
- we set up `"maxConcurrentActivityFunctions":1` in `host.json`, to stimulate instances spawning
- the orchestrator Function is in fan-out / fan-in pattern (`results =yield context.task_all(tasks)`)

**But there was only two instances that had spawned during the 3-4 hours run (we expected more).**

**We've read the documentation several times (and lot of GH issues) about scaling, but we're uncertain on what to try next to achieve the desired scaling behavior.**

![Capture d’écran 2021-02-02 à 10 09 15](https://user-images.githubusercontent.com/578154/106582462-4aad8780-6544-11eb-815a-6031b38375d0.png)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.