Azure / Azure/azure-functions-host
COMException and RpcException in Function App
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
We have had a couple of incidents recently where our function apps start erroring badly. We see a high percentage of failures (many 1000's) across all functions in the app caused by **System.Runtime.InteropServices.COMException** and **Microsoft.Azure.WebJobs.Script.Rpc.RpcException**.
The only way we have found to fix the issue is to restart the function app after which point the errors completely stop.
Interestingly, we have seen that this incident seems to affect all function apps running on a given app service plan.
The errors seem to be coming from the function runtime. It would be useful to know if this is caused by instability in the func runtime or if it is something we are doing wrong.
Our function apps are written in Node, run the latest V2 runtime and run on app service plans. Most of our functions are http triggers.
#### Investigative information
The last incident started on 2019-03-10 2100 UTC and lasted around 14 hours (at which point we restarted the app and the errors stopped)
Com Exception Example
- Timestamp: 2019-03-11T09:30:59.8876066Z
- Function App version: 2.0
- Invocation ID: 515cf6a6-79c9-46dc-bb62-db6fb1d02be1
- Region: SEA
Rpc Exception Example
- Timestamp: 2019-03-11T07:50:35.8048046Z
- Function App version: 2.0
- Invocation ID: 148bd6f4-2fda-4658-a446-48eca495451e
- Region: SEA
Contributor guide
Research direction
No source file, test, or entry point is named. Start by reviewing the reported COMException and RpcException events alongside the Function App and App Service plan details, then determine whether the host runtime or application behavior is responsible. Done means identifying a reproducible cause and documenting whether restarting the app is an expected mitigation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, node.js
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100