microsoft / microsoft/perfview

ETWReloggerTraceEventSource: access violation in tdh!CEventBase::Release on finalizer thread after Dispose

Open
#2,411 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
4.7k
Forks
775
Avg merge
5d 11h
Merged PRs (30d)
9

Description

Summary

After disposing an ETWReloggerTraceEventSource, the process intermittently crashes with an access violation in tdh!CEventBase::Release on the .NET finalizer thread. The crash is caused by the GC's RCW (Runtime Callable Wrapper) cleanup attempting to Release() a COM ITraceEvent object that has already been freed.

TraceEvent version

Microsoft.Diagnostics.Tracing.TraceEvent 3.2.2

Environment

  • Windows 11 (26100), x64
  • .NET 10

Reproduction

The crash is intermittent under normal conditions but becomes 100% reproducible when GFlags full page heap is enabled for the host process (gflags /p /enable MyApp.exe /full). Page heap turns the silent heap corruption into an immediate access violation at the point of the invalid memory access.

Minimal reproduction pattern:

// Run this repeatedly — crashes intermittently (deterministically with page heap)
for (int i = 0; i < 50; i++)
{
    using var relogger = new ETWReloggerTraceEventSource(inputEtlPath, outputEtlPath);
    relogger.Dynamic.All += ev =>
    {
        if (ShouldKeep(ev))
            relogger.WriteEvent(ev);
    };
    relogger.Process();
    // After the using block, the relogger is disposed.
    // At some later GC cycle, the finalizer thread crashes in tdh!CEventBase::Release.
}

Crash analysis

Full crash dump analysis via !analyze -v:

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_AVRF_c0000005_tdh.dll!CEventBase::Release
SYMBOL_NAME:        tdh!CEventBase::Release+21
EXCEPTION_CODE:     c0000005 (Access violation)

Stack trace (from the .NET finalizer thread):

ntdll!ExpInterlockedPushEntrySList+0xd     ← read from freed memory
tdh!CEventBase::Release+0x21              ← IUnknown::Release on freed ITraceEvent COM object
coreclr!RCW::ReleaseAllInterfaces+0x96    ← .NET RCW cleanup
coreclr!RCW::ReleaseAllInterfacesCallBack+0x58
coreclr!RCW::Cleanup+0x57
coreclr!RCWCleanupList::ReleaseRCWListRaw+0x16
coreclr!RCWCleanupList::ReleaseRCWListInCorrectCtx+0x76
coreclr!RCWCleanupList::CleanupAllWrappers+0xdf
coreclr!SyncBlockCache::CleanupSyncBlocks+0xf6
coreclr!DoExtraWorkForFinalizer+0x35
coreclr!FinalizerThread::FinalizerThreadWorker+0x134

Analysis

ETWReloggerTraceEventSource uses COM interop (CTraceRelogger, ITraceEvent) to wrap the Windows ITraceRelogger API. The library already does the right thing in several places:

  • ReloggerCallbacks.OnEvent calls Marshal.FinalReleaseComObject(source.m_curITraceEvent) after each event callback to eagerly release ITraceEvent COM objects (source)
  • Dispose(true) calls Marshal.FinalReleaseComObject(m_relogger) on the relogger itself (source)

However, the crash indicates that some ITraceEvent RCW objects survive past disposal and are later collected by the GC. When the finalizer thread runs RCW::ReleaseAllInterfaces, it calls Release() on the underlying tdh!CEventBase COM object, which has already been freed.

Possible escape paths for ITraceEvent RCWs that bypass the FinalReleaseComObject cleanup:

  1. ITraceEvent.Clone() at line 127 — creates a new COM object that is passed to Inject but may not be explicitly released
  2. m_relogger.CreateEventInstance() at line 148 and line 327 — creates new COM objects that may not be tracked
  3. Exception paths in OnEvent — if an exception occurs before FinalReleaseComObject is reached, m_curITraceEvent retains a reference that becomes a dangling RCW after the relogger is disposed

Workaround

Adding GC.Collect() + GC.WaitForPendingFinalizers() immediately after disposing the ETWReloggerTraceEventSource forces the RCW cleanup to happen in a controlled state. This appears to resolve the crash in our testing (5/5 iterations pass with page heap, vs 0/5 before the fix), though it is not a proper fix.

using var relogger = new ETWReloggerTraceEventSource(inputEtlPath, outputEtlPath);
relogger.Dynamic.All += ev => { /* ... */ };
relogger.Process();

// Workaround: flush finalizer queue to clean up orphaned ITraceEvent RCWs
// before the underlying COM objects become invalid
GC.Collect();
GC.WaitForPendingFinalizers();

Suggested fix

Ensure all ITraceEvent COM objects created during the relogger's lifetime are released via Marshal.FinalReleaseComObject before Dispose returns. Specifically:

  1. Release the return value of ITraceEvent.Clone() and CreateEventInstance() after Inject calls
  2. Wrap the OnEvent callback body in a try/finally to guarantee FinalReleaseComObject runs even if the user callback throws
  3. Consider calling GC.Collect + GC.WaitForPendingFinalizers at the end of Dispose(true) as a safety net

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/TraceEvent/ETWReloggerTraceEventSource.cs, especially Clone at line 127, CreateEventInstance at lines 148 and 327, OnEvent at line 437, and Dispose at line 202. Reproduce with the provided loop and GFlags full page heap, then trace each ITraceEvent lifetime and exception path. Done means the reproduction no longer crashes during finalizer cleanup without requiring forced GC.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
devtools, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.