Proposal for an Optimization Toggle for Real-time Mode to Reduce Memory Overhead Using RingBuffer

Hello PerfView Team,

I am working on a UE4 game project and currently utilizing the Microsoft.Windows.EventTracing library to parse stack information corresponding to samples and ContextSwitches recorded in ETL for the game process. My approach involves classifying stack information for each frame based on timestamps, ultimately yielding stack data for all threads in each frame. The results are as follows:

Currently, we use xperf to record the necessary performance data. Here is a snippet of the parsing code we use:

  IPendingResult<ISymbolDataSource> pendingSymbolData = trace.UseSymbols();
  IPendingResult<ICpuSampleDataSource> pendingCpuSamplingData = trace.UseCpuSamplingData();
  IPendingResult<ICpuSchedulingDataSource> myCpuSchedlingData = trace.UseCpuSchedulingData();
  var pendingProcessorCounters = trace.UseProcessorCounters();
  var pendingProcesses = trace.UseProcesses();
  var pendingMeteData = trace.UseMetadata();
  var pendingSystemInfo = trace.UseSystemMetadata();
  var pendingTraceStatistics = trace.UseTraceStatistics();
  trace.Process();
  
  ISymbolDataSource symbolData = pendingSymbolData.Result;
  ICpuSampleDataSource cpuSamplingData = pendingCpuSamplingData.Result;
  ISystemMetadata systemMetadata = pendingSystemInfo.Result;
  
  foreach (ICpuSample sample in cpuSamplingData.Samples)
  {
    if (sample.Stack != null)
    {
      //...get sample info
      foreach (var frameInfo in sample.Stack.Frames)
      {
          //... get callchin list
      }
    }
  }
  
  if (myCpuSchedlingData.HasResult)
  {
      foreach (ICpuThreadActivity slice in myCpuSchedlingData.Result.ThreadActivity)
      {
          if (slice.SwitchIn.Stack != null && 
              slice.WaitingDuration != null && 
              slice.Thread.Name != null)
          {
              
              //...get slice sample info
              foreach (var frameInfo in slice.SwitchIn.Stack.Frames)
              {
                  //... get callchin list
              }
          }
      }
  }

We are now looking to leverage ETW's Real-time mode for on-the-fly data recording and parsing. However, we've encountered a significant issue: if we enable the collection of ContextSwitch and Dispatcher stack information in a Microsoft.Diagnostics.Tracing.TraceEvent session, we observe a rapid increase in memory usage (more than 1+ MB/s), with no signs of stabilization or decrease.

  session.EnableKernelProvider(
      KernelTraceEventParser.Keywords.Profile
      | KernelTraceEventParser.Keywords.ContextSwitch
      | KernelTraceEventParser.Keywords.Dispatcher
      | KernelTraceEventParser.Keywords.Process
      | KernelTraceEventParser.Keywords.ImageLoad
      | KernelTraceEventParser.Keywords.Thread
                  ,
      KernelTraceEventParser.Keywords.Profile 
      | KernelTraceEventParser.Keywords.ContextSwitch
      | KernelTraceEventParser.Keywords.Dispatcher
      | KernelTraceEventParser.Keywords.Process
      | KernelTraceEventParser.Keywords.ImageLoad
      | KernelTraceEventParser.Keywords.Thread
  );

Upon investigating memory allocations with dotMemory, we noticed that most of the memory usage is concentrated in GrowableArray.

Would it be possible to implement a RingBuffer mechanism to store this data in Real-time Sessions? This feature could greatly optimize memory usage for real-time performance analysis, particularly in complex applications like ours.

Start by examining GrowableArray in the real-time Microsoft.Diagnostics.Tracing.TraceEvent path, using the ContextSwitch and Dispatcher stack collection shown in the issue. Review how TraceEvent sessions retain these events and determine the intended ring-buffer retention behavior. Done means a documented implementation approach with validated memory usage under real-time collection.