Profiling Build on Windows Server 2022 has missing CPU Sampling Data

When enabling profiling with WPR on my Windows Server 2022 Runner I get no CPU sampling data when running on AMD Ryzen CPUs.

To Reproduce

1. Execute Github Action on Windows

 jobs:
    build:
        runs-on: windows-latest

3. Make sure you run on an AMD EPYC 7763 64-Core Processor
4. wpr -start CPU
5. wpr -stop c:\temp\NoCpuSamplingData.etl

Expected behavior
The recorded data should contain Context Switch and CPU sampling data. But CPU sampling data is
not recorded.

Runner Version and Platform

Windows Server 2022

What's not working?

See
https://github.com/Siemens-Healthineers/ETWAnalyzer/suites/10408928846/artifacts/514249781
for an example with no CPU sampling data.
A good example
https://github.com/Siemens-Healthineers/ETWAnalyzer/suites/10433849531/artifacts/516059225

This is a long standing bug of some device driver which tinkers with MSRs and breaks CPU sampling. Please check your drivers which are loaded in Server2022.

One of the following non MS drivers should be responsible for this behavior:

C:\GitHubActionCPUSamplingDataMissing>etwanalyzer -fd TestExecution_144294msfv-az420-830.20230117-070058.json -dump version -dll *.sys -versionfilter !*Microsoft* 
1/17/2023 7:02:38 AM   TestExecution_144294msfv-az420-830.20230117-070058 
  System(4)            
    3ware.sys C:\Windows\system32\drivers FileVersion: 5.1.0.51, VersionString: 5.01.00.051, ProductVersion: WindowsBlue, ProductName: LSI 3ware RAID Controller, Description: LSI 3ware SCSI Storport Driver,
    ADP80XX.SYS C:\Windows\system32\drivers FileVersion: 1.3.0.10769, VersionString: 1.3.0.10769 (NT.150223-1854), ProductName: PMC-Sierra HBA Controller, Description: PMC-Sierra Storport  Driver For SPC8x6G SAS/SATA controller,
    amdsata.sys C:\Windows\system32\drivers FileVersion: 1.1.3.277, VersionString: 1.1.3.277 (NT.150223-1854), ProductName: AHCI 1.3 Device Driver, Description: AHCI 1.3 Device Driver,
    amdsbs.sys C:\Windows\system32\drivers FileVersion: 3.7.1540.43, VersionString: 3.7.1540.43 (PART3.121207-1123), ProductName: AMD Technology AHCI Compatible Controller, Description: AMD Technology AHCI Compatible Controller Driver for Windows - AMD64 platform,
    amdxata.sys C:\Windows\system32\drivers FileVersion: 1.1.3.277, VersionString: 1.1.3.277 (NT.150223-1854), ProductName: Storage Filter Driver, Description: Storage Filter Driver,
    arcsas.sys C:\Windows\system32\drivers FileVersion: 7.5.0.32048, VersionString: 7.5.0.32048 (NT.150223-1854), ProductName: Adaptec RAID Controller, Description: Adaptec SAS RAID WS03 Driver,
    bfadfcoei.sys C:\Windows\system32\drivers FileVersion: 10.0.10060.0, VersionString: 3.2.26.1, ProductName: QLogic BR-series FC/FCoE HBA Stor Miniport Driver, Description: QLogic BR-series FC/FCoE HBA Stor Miniport Driver,
    bfadi.sys C:\Windows\system32\drivers FileVersion: 10.0.10060.0, VersionString: 3.2.26.1, ProductName: QLogic BR-series FC/FCoE HBA Stor Miniport Driver, Description: QLogic BR-series FC/FCoE HBA Stor Miniport Driver,
    bxfcoe.sys C:\Windows\system32\drivers 7.14.22.101 ProductName: QLogic FCoE Offload driver, Description: FCoE Offload x64 FREE,
    bxois.sys C:\Windows\system32\drivers 7.14.16.101 ProductName: QLogic iSCSI offload driver, Description: iSCSI offload x64 FREE (inbox),
    bxvbda.sys C:\Windows\system32\drivers 7.12.31.105 ProductName: QLogic Gigabit Ethernet, Description: QLogic Gigabit Ethernet VBD,
    cht4sx64.sys C:\Windows\system32\drivers 6.11.4.100 ProductVersion: 10.0.10011.16384, ProductName: Chelsio Communications iSCSI Controller, Description: Chelsio iSCSI VMiniport Driver,
    elxfcoe.sys C:\Windows\system32\drivers FileVersion: 11.0.247.8000, VersionString: 11.0.247.8000 01/26/2016 WS2K12 64 bit x64, ProductName: Emulex WS2K12 Storport Miniport Driver x64, Description: Emulex Storport Miniport Driver,
    elxstor.sys C:\Windows\system32\drivers FileVersion: 12.6.165.8000, VersionString: 12.6.165.8000 12/07/2019 WS2K12 64 bit x64, ProductName: Emulex WS2K12 Storport Miniport Driver x64, Description: Emulex Storport Miniport Driver,
    evbd0a.sys C:\Windows\system32\drivers 7.13.65.105 ProductName: QLogic 10 GigE, Description: QLogic 10 GigE VBD,
    evbda.sys C:\Windows\system32\drivers 7.13.171.102 ProductName: QLogic 10 GigE, Description: QLogic 10 GigE VBD,
    HpSAMD.sys C:\Windows\system32\drivers FileVersion: 8.0.4.0, VersionString: 8.0.4.0 Build 1 Media Driver (x86-64) (PART_L3.130128-0938), ProductVersion: 8.0.4.0 Build 1 Media Driver (x86-64), ProductName: Smart Array SAS/SATA Controller Media Driver, Description: Smart Array SAS/SATA Controller Media Driver,
    iaStorAVC.sys C:\Windows\system32\drivers 15.44.0.1015 ProductName: Intel(R) Rapid Storage Technology driver (inbox), Description: Intel(R) Rapid Storage Technology driver (inbox) - x64,
    iaStorV.sys C:\Windows\system32\drivers 8.6.2.1019 ProductName: Intel Matrix Storage Manager driver, Description: Intel Matrix Storage Manager driver - x64,
    ItSas35i.sys C:\Windows\system32\drivers 2.61.19.80 ProductVersion: 10.0.10011.16384, ProductName: Windows (R) Win 7 DDK driver, Description: Avago SAS Gen3.5 Driver (StorPort),
    lsi_sas.sys C:\Windows\system32\drivers FileVersion: 1.34.3.83, VersionString: 1.34.03.83 (NT.150223-1854), ProductVersion: 1.34.03.83, ProductName: LSI Fusion-MPT SAS Driver (StorPort), Description: LSI Fusion-MPT SAS Driver (StorPort),
    lsi_sas2i.sys C:\Windows\system32\drivers FileVersion: 2.0.79.82, VersionString: 2.00.79.82, ProductVersion: 10.0.10011.16384, ProductName: Windows (R) Win 7 DDK driver, Description: LSI SAS Gen2 Driver (StorPort),
    lsi_sas3i.sys C:\Windows\system32\drivers 2.51.27.80 ProductVersion: 10.0.10011.16384, ProductName: Windows (R) Win 7 DDK driver, Description: Avago SAS Gen3 Driver (StorPort),
    MegaSas2i.sys C:\Windows\system32\drivers FileVersion: 6.714.22.0, VersionString: 6.714.22.00, ProductName: MEGASAS2i RAID Controller Driver for Windows, Description: MEGASAS2i RAID Controller Driver for Windows,
    megasas35i.sys C:\Windows\system32\drivers FileVersion: 7.716.2.0, VersionString: 7.716.02.00, ProductName: MEGASAS RAID Controller Driver for Windows, Description: MEGASAS RAID Controller Driver for Windows,
    megasr.sys C:\Windows\system32\drivers FileVersion: 15.2.2013.129, VersionString: 15.02.2013.0129, ProductName: MegaRAID Software RAID, Description: LSI MegaRAID Software RAID Driver,
    mpi3drvi.sys C:\Windows\system32\drivers FileVersion: 3.0.71.90, VersionString: 3.00.71.90, ProductVersion: 10.0.10011.16384, ProductName: Windows (R) Win 7 DDK driver, Description: Broadcom MPI 3.0 Driver (StorPort),
    mvumis.sys C:\Windows\system32\drivers FileVersion: 1.0.5.1016, VersionString: 1.0.5.1016  (PART_L3.131021-1012), ProductVersion: 1.0.5.1016, ProductName: Marvell Flash Controller, Description: Marvell Flash Controller Driver,
    nvraid.sys C:\Windows\system32\drivers FileVersion: 10.6.0.23, VersionString: 10.6.0.23 (PART_L3.131021-1012), ProductName: NVIDIA nForce(TM) RAID Driver, Description: NVIDIA® nForce(TM) RAID Driver,
    nvstor.sys C:\Windows\system32\drivers FileVersion: 10.6.0.23, VersionString: 10.6.0.23 (PART_L3.131021-1012), ProductName: NVIDIA nForce(TM) SATA Driver, Description: NVIDIA® nForce(TM) Sata Performance Driver,
    percsas2i.sys C:\Windows\system32\drivers FileVersion: 6.805.3.0, VersionString: 6.805.03.00 (rs1_onecore_stacksp_sfs4(paudina).160302-1205), ProductVersion: 6.805.03.00, ProductName: MEGASAS RAID Controller Driver for Windows, Description: MEGASAS RAID Controller Driver for Windows,
    percsas3i.sys C:\Windows\system32\drivers FileVersion: 6.604.6.0, VersionString: 6.604.06.00, ProductName: MEGASAS RAID Controller Driver for Windows, Description: MEGASAS RAID Controller Driver for Windows,
    pvscsii.sys C:\Windows\system32\drivers 1.3.15.0 ProductVersion: 1.3.15.0 build-18052479, ProductName: VMware PVSCSI StorPort driver (64-bit), Description: VMware PVSCSI StorPort driver (64-bit),
    qefcoe.sys C:\Windows\system32\drivers 8.42.1.101 ProductName: QLogic FCoE offload driver, Description: FCoE offload x64 FREE,
    qeois.sys C:\Windows\system32\drivers 8.42.2.101 ProductName: QLogic iSCSI offload driver, Description: iSCSI offload x64 FREE,
    qevbda.sys C:\Windows\system32\drivers 8.45.1.103 ProductName: QLogic FastLinQ Ethernet, Description: QLogic FastLinQ Ethernet VBD,
    ql2300i.sys C:\Windows\system32\drivers 9.4.2.103 ProductName: QLogic Fibre Channel Stor Miniport Driver, Description: QLogic Fibre Channel Stor Miniport Driver,
    ql40xx2i.sys C:\Windows\system32\drivers FileVersion: 2.1.5.0, VersionString: 2.1.5.0 (STOREx wx64), ProductName: QLA40XX iSCSI Host Bus Adapter, Description: QLogic iSCSI Storport Miniport Inbox Driver,
    qlfcoei.sys C:\Windows\system32\drivers 9.1.11.3 ProductName: QLogic FCoE Stor Miniport Inbox Driver, Description: QLogic FCoE Stor Miniport Inbox Driver,
    SmartSAMD.sys C:\Windows\system32\drivers 1.50.1.0 ProductName: SmartRAID. SmartHBA PQI Storport Driver, Description: Storport Miniport Driver for SmartRAID/SmartHBA Controllers,
    stexstor.sys C:\Windows\system32\drivers FileVersion: 5.1.0.10, VersionString: 5.1.0000.10 (PART.121116-1330), ProductVersion: 5.1.0000.10, ProductName: Promise® SuperTrak EX Series, Description: Promise SuperTrak EX Series Driver for Windows x64,
    vsmraid.sys C:\Windows\system32\drivers 7.0.9600.6352 ProductName: VIA RAID driver, Description: VIA RAID DRIVER FOR AMD-X86-64,
    vstxraid.sys C:\Windows\system32\drivers 8.0.9200.8110 ProductName: VIA StorX RAID Controller Driver, Description: VIA StorX RAID Controller Driver,

After doing a diff only three drivers are left

amdppm.sys

FileVersion: 10.0.20348.887, VersionString: 10.0.20348.887 (WinBuild.160101.0800), ProductName: Microsoft® Windows® Operating System, Description: Processor Device Driver,

Or

hvservice.sys

FileVersion: 10.0.20348.1311, VersionString: 10.0.20348.1311 (WinBuild.160101.0800), ProductName: Microsoft® Windows® Operating System, Description: Hypervisor Boot Driver,

vmbusr.sys

FileVersion: 10.0.20348.469, VersionString: 10.0.20348.469 (WinBuild.160101.0800), ProductName: Microsoft® Windows® Operating System, Description: Microsoft Hyper-V Virtual Machine Bus Root Driver,

which could be interesting as well. One of these two drivers breaks CPU sampling during boot. Please check which one it is and get a fixed version from the corresponding vendor.

Start by reproducing the WPR commands on a Windows Server 2022 AMD EPYC runner and compare the resulting ETL files with the linked good and bad artifacts. Investigate the listed drivers, especially amdppm.sys, hvservice.sys, and vmbusr.sys, to identify which prevents CPU sampling during boot. Done means the responsible driver is confirmed and a fixed vendor version is identified.