Azure / Azure/azure-functions-host
Likely race condition in FileTraceWriter
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 38
Description
A user on Azure Functions doing a high throughput event hubs scenario reported they often see an exception with the following trace:
```
System.ArgumentOutOfRangeException: Index and count must refer to a location within the buffer. Parameter name: chars at
System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32
charCount, Byte[] bytes, Int32 byteIndex, Boolean flush) at
System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at
System.IO.StreamWriter.Flush() at
Microsoft.Azure.WebJobs.Host.CompositeTraceWriter.Flush() at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task
task) at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.d__12.MoveNext()
```
(original report [here](https://social.msdn.microsoft.com/Forums/en-US/baaa5a7b-c662-4909-b80c-21274ea0e1a6/argumentoutofrangeexception-in-eventhub-triggered-function?forum=AzureFunctions))
From the trace it looks likely that the problem is unsynchronized access to a StreamWriter. Considering that FileTraceWriter has the only non-trivial implementation of TraceWriter.Flush, its the most likely source of the issue.
Contributor guide
Research direction
Start by locating FileTraceWriter and its TraceWriter.Flush implementation, then inspect how its StreamWriter is accessed during concurrent function execution. Reproduce the reported flush exception under concurrent writes and verify that the failure no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100