Stack Overflow Exception caused by very deep parent/child Task chains

Start with Task.cs at FinishStageTwo and ProcessChildCompletion, then inspect the related EventSource path shown through Interop.cs. Compare the behavior against the recent source and NuGet changes and use a reproducible crash or diagnostic dump to isolate the deep parent/child chain. Done means identifying the cause of the recursive completion path and preventing the StackOverflowException with regression coverage.

Description

I have a .NET 9 ASP.NET application running as a service. After the latest upgrades of nuget packages, the application crashes periodically with a StackOverflow exception

I tried to repro it but I couldn't. So I attached Visual Studio Debugger and let it run until it happened again.

The call stack does not touch any application code

and looks confined into System.Private.CoreLib.DLL

[Managed to Native Transition]
System.Private.CoreLib.dll!Interop.Advapi32.EventWriteTransfer(long registrationHandle, System.Diagnostics.Tracing.EventDescriptor eventDescriptor, System.Guid* activityId, System.Guid* relatedActivityId, int userDataCount, System.Diagnostics.Tracing.EventProvider.EventData* userData) Line 3853
	at -\Interop.cs(3853)
System.Private.CoreLib.dll!System.Diagnostics.Tracing.EventSource.WriteEventWithRelatedActivityIdCore(int eventId, System.Guid* relatedActivityId, int eventDataCount, System.Diagnostics.Tracing.EventSource.EventData* data) Line 1126
	at System.Diagnostics.Tracing\EventSource.cs(1126)
System.Private.CoreLib.dll!System.Diagnostics.Tracing.EventSource.WriteEvent(int eventId, int arg1, int arg2) Line 681
	at System.Diagnostics.Tracing\EventSource.cs(681)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2154
	at System.Threading.Tasks\Task.cs(2154)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
[The 2 frame(s) above this were repeated 6033 times]
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
.
.
.
.
.
.
.
.
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ProcessChildCompletion(System.Threading.Tasks.Task childTask) Line 2210
	at System.Threading.Tasks\Task.cs(2210)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishStageTwo() Line 2166
	at System.Threading.Tasks\Task.cs(2166)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishSlow(bool userDelegateExecute) Line 2098
	at System.Threading.Tasks\Task.cs(2098)
System.Private.CoreLib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot, System.Threading.Thread threadPoolThread) Line 2328
	at System.Threading.Tasks\Task.cs(2328)
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 789
	at System.Threading\ThreadPoolWorkQueue.cs(789)
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Line 1175
	at System.Threading\PortableThreadPool.cs(1175)

This is the Visual studio screen when it occurs

and this is the github pilot comment on it

This error happens because a System.StackOverflowException is being thrown, indicating that the program has exhausted the stack memory due to excessive recursion or an infinite loop.
This might be happening because the FinishStageTwo method is being called recursively without a proper termination condition. The stack trace shows that FinishStageTwo calls ProcessChildCompletion, which in turn calls FinishStageTwo again, creating a loop that eventually leads to a stack overflow. This recursion is repeated thousands of times, as indicated by the repeated stack frames.
To fix this issue, you need to ensure that the recursion has a proper base case to terminate. Check the logic in FinishStageTwo and ProcessChildCompletion to ensure that they do not call each other indefinitely. You may also want to add logging to understand the flow of execution better and identify the conditions under which the recursion occurs.
Additional information needed to solve it:
•	The complete implementation of FinishStageTwo, ProcessChildCompletion, and related methods.
•	The context in which these methods are being called.
•	Any custom logic or event handling that might be influencing the recursion.

In a previous occurrence, I had also collected a WER crash dump, but it didn't help me much in troubleshooting. The file compressed is 125 MB, so it cannot be attached here.

Reproduction Steps

Cannot reproduce. Only thing I can do is let it run until it crashes

Expected behavior

It shouldn't crash

Actual behavior

It crashes (sometimes after 10 minutes other times it takes many hours

Regression?

The application did run correctly in the past 6 months. A new deployment has been released few days ago and it contains multiple source changes and multiple nuget upgrades. So it's not easy to point to a .NET update that could have brought to this

Known Workarounds

No workarounds

Configuration

C:\Tools\procdump>dotnet version

Welcome to .NET 9.0!
---------------------
SDK Version: 9.0.200

Windows11 (10.0.22631)
x64
Do not know whether it is specific to this configuration

Other information

No response