BSOD (MULTIPLE_IRP_COMPLETE_REQUESTS, SYNTHETIC_WATCHDOG_TIMEOUT) when using CimFS snapshotter on Windows Server 2022
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 694
- Forks
- 304
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 28
Description
Describe the bug
Using the CimFS snapshotter with containerd 2.2.1 on Windows Server 2022 (build 20348.4773, KB5075906 Feb 2026 cumulative update) causes repeated BSODs under container workload. Two distinct bugchecks occurred within 13 minutes of enabling CimFS:
0x00000044(MULTIPLE_IRP_COMPLETE_REQUESTS). a driver attempted to complete an IRP that was already completed.0x000001ca(SYNTHETIC_WATCHDOG_TIMEOUT). system became unresponsive.
Both crashes happened while Kubernetes was pulling and starting ~15-20 container images simultaneously using the CimFS snapshotter. The system was stable before switching from the windows snapshotter to cimfs and stable again after reverting.
To Reproduce
- Install containerd v2.2.1 on Windows Server 2022 (build 20348.4773).
- Configure containerd to use the
cimfssnapshotter:version = 3 [plugins.'io.containerd.cri.v1.images'] snapshotter = 'cimfs' [plugins.'io.containerd.transfer.v1.local'] [[plugins.'io.containerd.transfer.v1.local'.unpack_config]] platform = 'windows/amd64' snapshotter = 'cimfs' differ = 'cimfs' - Start containerd and kubelet.
- Pull multiple Windows container images concurrently (~15-20 images pulled by Kubernetes daemonsets and workloads).
- System BSODs within minutes.
Expected behavior
The CimFS snapshotter should handle concurrent image pulls and container creation without causing kernel crashes.
Configuration:
- Edition: Windows Server 2022 Datacenter (21H2)
- OS Build: 20348.4773 (KB5075906, February 2026 cumulative update)
cimfs.sysdriver version: 10.0.20348.3804- Container engine: containerd v2.2.1
- Container runtime shim: containerd-shim-runhcs-v1 (bundled with containerd 2.2.1)
- Orchestrator: Kubernetes v1.34.2 (kubelet)
- Hardware: Bare metal, AMD Ryzen 9 7950X, 64GB RAM
- Base images: Various
mcr.microsoft.com/windows/servercore:ltsc2022andmcr.microsoft.com/windows/nanoserver:ltsc2022based images
Additional context
Bugcheck details from Windows Event Log:
Bugcheck 1 (12:37:07 PST):
Code: 0x00000044 (MULTIPLE_IRP_COMPLETE_REQUESTS)
Parameters: (0xffffb1075846f4c0, 0x000000000000125a, 0x0000000000000000, 0x0000000000000000)
Minidump: 030526-8984-01.dmp (1.4 MB)
Bugcheck 2 (12:50:18 PST):
Code: 0x000001ca (SYNTHETIC_WATCHDOG_TIMEOUT)
Parameters: (0x0000000019628f83, 0x00000001bf799269, 0x00000001bf7ca01d, 0x0000000000000008)
Minidump: 030526-9125-01.dmp (630 KB)
Timeline:
- Switched containerd snapshotter from
windowstocimfsand restarted services - Kubernetes began pulling ~15-20 container images concurrently (daemonset pods + workloads)
- CimFS snapshots were being created successfully (55+ active snapshots observed)
- Several containers started successfully with CimFS layers before the crash
- First BSOD at 12:37 (
MULTIPLE_IRP_COMPLETE_REQUESTS) - System rebooted, containers began re-pulling images
- Second BSOD at 12:50 (
SYNTHETIC_WATCHDOG_TIMEOUT) - Reverted to
windowssnapshotter. system has been stable since
The CimFS snapshotter was loading correctly (io.containerd.snapshotter.v1 cimfs ok, io.containerd.differ.v1 cimfs ok) and creating snapshots before the crashes. The issue appears to be in the cimfs.sys kernel driver under concurrent load.
No prior bugchecks on this system since November 2022 (over 3 years of stability). Minidumps are available if needed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported containerd 2.2.1 and CimFS configuration under concurrent Windows image pulls, then review the available minidumps and hcsshim's CimFS-related entry points. Done means determining whether the failure is actionable in hcsshim or belongs to the Windows CimFS driver, with a confirmed reproduction and diagnosis documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100