NotImplementedException in SystemTargetBasedMembershipTable
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 14h 42m
- Merged PRs (30d)
- 354
Description
I sometimes hit the following line during local testing on my machine with a development localhost cluster:
https://github.com/dotnet/orleans/blob/8035ed932d0785950410cee348bb2a3d3c41b81d/src/Orleans.Runtime/MembershipService/SystemTargetBasedMembershipTable.cs#L110
I could not yet figure out which situations lead to this behavior, as it only happens sometimes.
I assume, since it is in-memory, that a simple ``return Task.CompletedTask`` would be sufficient here. It should basically be a no-op for a 1-node localhost cluster, right?
Here is a callstack:
``> Orleans.Runtime.dll!Orleans.Runtime.MembershipService.SystemTargetBasedMembershipTable.CleanupDefunctSiloEntries(System.DateTimeOffset beforeDate) Line 110 C#
Orleans.Runtime.dll!Orleans.Runtime.MembershipService.MembershipTableCleanupAgent.CleanupDefunctSilos() Line 71 C#
[Resuming Async Method]
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.AsyncStateMachineBoxd__6>.ExecutionContextCallback(object s) Line 131 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 138 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.AsyncStateMachineBoxd__6>.MoveNext(System.Threading.Thread threadPoolThread) Line 163 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.AsyncStateMachineBoxd__6>.MoveNext() Line 146 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 140 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 162 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 2850 C#
Orleans.Runtime.dll!Orleans.Runtime.AsyncTimer.NextTick(System.TimeSpan? overrideDelay) Line 88 C#
[Resuming Async Method]
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 138 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.AsyncStateMachineBoxd__8>.MoveNext(System.Threading.Thread threadPoolThread) Line 163 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 140 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 162 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 2850 C#
System.Private.CoreLib.dll!System.Threading.Tasks.TaskFactory.CompleteOnInvokePromise.Invoke(System.Threading.Tasks.Task completingTask) Line 1000 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 2741 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.DelayPromise.CompleteTimedOut() Line 679 C#
System.Private.CoreLib.dll!System.Threading.TimerQueueTimer.Fire(bool isThreadPool) Line 259 C#
System.Private.CoreLib.dll!System.Threading.TimerQueue.FireNextTimers() Line 218 C#
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 690 C#
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Line 1328 C#
[Async Call Stack]
[Async] System.Private.CoreLib.dll!System.Threading.Tasks.Task.Run C#
``
Contributor guide
Research direction
Start at src/Orleans.Runtime/MembershipService/SystemTargetBasedMembershipTable.cs line 110 and trace the call from MembershipTableCleanupAgent.CleanupDefunctSilos using the provided call stack. Reproduce the intermittent failure with a development localhost cluster, determine when cleanup is invoked, and verify that the one-node in-memory membership table completes cleanup without throwing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100