Azure / Azure/azure-functions-core-tools

Unexpected Python Timeout Behaviour

Open
#3,696 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
1.5k
Forks
498
Avg merge
4d 20h
Merged PRs (30d)
14

Description

Hi, I am trying to understand how exactly the timeouts are handled in python based azure functions. We are running the following scenario:

- running "func start" on a virtual machine with process count == 8 and thread count == 12 and a timeout of 10 minutes
- running a queue based function running with python 3.8
- messages are processed concurrently and using a shared threadlock for thread unsafe objects (all locks are used in a with context)

The issue that we are seeing is that if one thread of a process runs into a timeout that thread is killed, as expected, but all other threads of that process stay open and running until their respective timeout is also reached. They do not finish processing their messages and seem to be deadlocked or sleeping.

My question is the following, what is actually going on here? Why does the azure function worker not kill the process directly and end all threads? Why does the thread that is killed not release the lock properly?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the queue-function scenario with `func start`, Python 3.8, eight processes, twelve threads, and the ten-minute timeout described in the issue. Observe the other threads and shared locks when one thread times out; the work is done when the timeout, process, thread, and lock behavior is explained from the reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.