Azure / Azure/azure-functions-host

On startup sometimes language python worker crashes the process.

Open
#3,552 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 11h
Merged PRs (30d)
37

Description

Repro steps:

```
mkdir py-func
cd py-func
python3 -m venv venv
source venv/bin/activate
func init . --worker-runtime python
pip install azure-functions==1.0.0a4 azure-functions-worker==1.0.0a4 grpcio==1.14.2 grpcio-tools==1.14.2 protobuf==3.6.1 six==1.11.0 tensorflow==1.5.0 pillow==5.0.0 numpy==1.14.1 flask urllib3
func new --name "Http Trigger"
func start
```

It usually works, but every now and then it'll crash with this exception:

```
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.Collections.Generic.Queue`1.Enumerator.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
at System.String.Join(String separator, IEnumerable`1 values)
at Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerChannel.OnProcessExited(Object sender, EventArgs e) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Rpc\LanguageWorkerChannel.cs:line 184
at Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerChannel.b__34_2(Object sender, EventArgs e) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Rpc\LanguageWorkerChannel.cs:line 154
at System.Diagnostics.Process.OnExited()
at System.Diagnostics.Process.RaiseOnExited()
at System.Diagnostics.Process.CompletionCallback(Object waitHandleContext, Boolean wasSignaled)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
```

Contributor guide

Open the contributing guide

Research direction

Start with src/WebJobs.Script/Rpc/LanguageWorkerChannel.cs around lines 154 and 184, then run the documented Python worker reproduction. Trace the process-exit handling and the reported collection-enumeration exception. Done means the startup scenario no longer crashes the host with this exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.