Azure / Azure/azure-functions-host
Python language worker crashes when executing functions
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 38
Description
All requests to the container failed with the below exception.
West US
FunctionsLogs
| where RoleInstance contains "171AE528-636740280065455456"
| where TIMESTAMP >= datetime(2018-10-01 23:23:00.0000000)
| where TIMESTAMP <= datetime(2018-10-01 23:23:00.0000000) + 30m
| where Tenant == "fe133483000c4137a26aabfc6365308c"
| where SourceNamespace == "LWAWS"
| where Level < 4
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.scalethis ---\u003e Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerChannelException : Failed to start language worker for: python ---\u003e System.AggregateException : One or more errors occurred. (bash exited with code 137 ) (bash exited with code 137 ) (bash exited with code 137 ) ---\u003e bash exited with code 137 ---\u003e (Inner Exception #0) Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerProcessExitException : bash exited with code 137 \u003c--- ---\u003e (Inner Exception #1) Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerProcessExitException : bash exited with code 137 \u003c--- ---\u003e (Inner Exception #2) Microsoft.Azure.WebJobs.Script.Rpc.LanguageWorkerProcessExitException : bash exited with code 137 \u003c--- End of inner exception
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerLanguageInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/Rpc/WorkerLanguageInvoker.cs : 74
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 84
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task`1 src) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionGenerator.cs : 225
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionInvoker.cs : 63
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionInvoker.cs : 63
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task`1 src) at /azure-functions-host-26390c79fe40b04617bc7fc3a9add054dec27310/src/WebJobs.Script/Description/FunctionGenerator.cs : 225
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance,ParameterHelper parameterHelper,ILogger logger,CancellationTokenSource functionCancellationTokenSource) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs : 508
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs : 444
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs : 249 End of inner exception
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs : 293
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken) at C:\\projects\\azure-webjobs-sdk-rqm4t\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs : 89
Contributor guide
Research direction
Start with the Python language-worker startup path in WorkerLanguageInvoker.cs and FunctionInvokerBase, using the supplied FunctionsLogs query and the repeated bash exit-code-137 trace as the failure signal. Trace the request through FunctionGenerator and determine what allows Python function execution to complete without the worker crashing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100