Azure / Azure/azure-functions-host

Potential TimerTrigger deadlock on Elastic Premium plans

Open
#7,545 0 comments 0 reactions 0 assignees View on GitHub
extensions: timer needs-investigation
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

_Just creating this issue for tracking, and to follow-up on an internal incident_

We recently investigated an incident where a customer's TimerTrigger Function stopped executing until the application restarted. Before the restart, the last invocation of the TimerTrigger resulted in a time-out, which led the language worker to restart **but the Host remained online**. It appears that since the Host remained online, the TimerTrigger lease for the failed Function never got released; this seems to be the reason why the TimerTrigger stopped executing until the restart took place. When the restart took place, our logs indicate that the TimerTrigger lease finally got released and so, after the restart completed, the Function continued executing as expected.

The plan was Elastic Premium, the language was PowerShell.

After discussing with @fabiocav, we believe a simple reproducer can be written as follows:
(1) Create a TimerTrigger Function "A" to run every minute
(2) Make "A" pause for "5 minutes"
(3) Force the TimerTrigger timeout to be 2 minutes
This specific configuration should replicate, on a smaller total time window, the sequence of events we observed on the incident.

Then, observe what code path is taken and ensure the lease is released.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported sequence with a PowerShell TimerTrigger that runs every minute, pauses for five minutes, and has a two-minute timeout on an Elastic Premium plan. Trace the TimerTrigger timeout and lease-handling path, then confirm that the lease is released while the Host remains online and the function can execute again without an application restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp, powershell
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.