Azure / Azure/azure-functions-host

Dependency Injection slow when running in dedicated App Service Plan

Open
#9,497 5 comments 2 reactions 0 assignees View on GitHub
area: dependency injection needs-investigation
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 10h
Merged PRs (30d)
36

Description

When running our function app in a dedicated App Service Plan, the dependency injection resolving services is very slow and causes high CPU usage. When running in a consumption plan, it is a lot faster.

#### Investigative information

We have recently migrated some Webjobs to a Functions app. This is running in a dedicated App Service Plan. Most of the functions are queue triggered, with 1 timer trigger (every 30 seconds).
What we are seeing now, is that the function executions take a lot longer than expected and cause high CPU usage during execution.
For example the timer trigger, which (most of the time) only does 1 database call that takes 1-2 ms, takes 700-1000 ms on average. For example (as seen in Application Insights):

![image](https://github.com/Azure/azure-functions-host/assets/2334483/4395b1dc-8ca5-4a79-8b11-98ed01f4fbf2)

Invocation ID: a423fa6d-07b5-4fa3-a4d7-f4f100882f2f
Timestamp: 2023-08-30T07:52:00.0100831+00:00
Region: West Europe

The same pattern happens in our test environment as well, which is a different app service and plan.

Using the profiler I have traced this back to the dependency injection resolving services.
I have created a test project, with just 1 timer function and 24 empty classes to resolve (see below for code).
This uses the IServiceProvider to be able to measure resolving the service with application insights.

Running in a dedicated app service plan (with no other apps) on plan B1, this is the result:
2023-08-30 - Function timing App Service Plan B1

Running in a dedicated app service plan (with no other apps) on the highest possible plan (Premium v2 P3V2), it is twice as fast, but still takes 20ms:
2023-08-30 - Function timing App Service Plan Premium v2 P3V2

Running in a consumption plan, it suddenly only takes 1-3ms:
2023-08-30 - Function timing App Service Consumption plan

According to [this table](https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale) the consumption plan runs on just 100 ACU and 1.5 GB memory per instance, so it does not have more resources compared to the dedicated plans.

I cannot explain these differences, especially in our production environment. Is there something I could do to improve this?

I did come across issue #8073, could that be related?

#### Repro steps

Everything is running .NET 6 and Functions runtime version 4.
Repo: https://github.com/Holmwoods/azure-functions-dependency-injection-performance

```csharp
public class SendEmail
{
private readonly IServiceProvider _serviceProvider;
private readonly TelemetryClient _telemetry;

public SendEmail(IServiceProvider provider)
{
_serviceProvider = provider;
_telemetry = provider.GetService();
}

[FunctionName("SendEmail")]
public void Run([TimerTrigger("*/30 * * * * *")] TimerInfo myTimer, ILogger log)
{
using (var operation = _telemetry.StartOperation("Get Service1"))
{
_serviceProvider.GetService();
}
using (var operation = _telemetry.StartOperation("Get Service2"))
{
_serviceProvider.GetService();
}
}
}

internal class Service1
{
public Service1(Service2 service2, Service3 service3, Service4 service4) { }
}
internal class Service2
{
public Service2(Service3 service3, Service4 service4) { }
}
internal class Service3
{
public Service3(Service4 service4) { }
}
internal class Service4
{
public Service4(Service5 service5, Service6 service6, Service7 service7) { }
}
internal class Service5
{
public Service5(Service8 service8, Service9 service9, Service10 service10, Service11 service11, Service12 service12, Service13 service13, Service14 service14) { }
}
internal class Service6
{
public Service6(Service15 service15, Service16 service16, Service17 service17, Service18 service18, Service19 service19, Service20 service20, Service21 service21, Service22 service22, Service23 service23, Service24 service24) { }
}
internal class Service7
{
}
internal class Service8
{
}
internal class Service9
{
}
internal class Service10
{
}
internal class Service11
{
}
internal class Service12
{
}
internal class Service13
{
}
internal class Service14
{
}
internal class Service15
{
}
internal class Service16
{
}
internal class Service17
{
}
internal class Service18
{
}
internal class Service19
{
}
internal class Service20
{
}
internal class Service21
{
}
internal class Service22
{
}
internal class Service23
{
}
internal class Service24
{
}
```

Contributor guide

Open the contributing guide

Research direction

Start by running the linked Holmwoods/azure-functions-dependency-injection-performance reproduction with its .NET 6 timer function and IServiceProvider resolutions. Compare the timing and CPU behavior across dedicated and consumption plans, then review issue #8073 for related context. Done means identifying a host-side cause or documenting a concrete mitigation or fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
backend, cloud, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.