[Kernel Proposal] Prevent VMBus Control-Plane Starvation & High-Order Ring Allocation Deadlocks Under Memory Pressure
- Dominant language
- C++
- Stars
- 33.7k
- Forks
- 1.8k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 116
Description
### Problem Statement & Architectural Context
Under heavy memory allocation (dense container compilation, heavy memory tiering, or active swap pressure), Microsoft WSL2 instances frequently encounter severe control-plane timeouts:
1. **HCS Watchdog Timeout:** When guest available memory drops into direct reclaim, atomic page allocations (`GFP_ATOMIC`) required by synthetic VMBus packet rings and host heartbeat channels fail. The Windows Host Compute System infers a hard lock and terminates the instance with `Wsl/Service/E_UNEXPECTED (0x8000ffff)` or drops the virtual switch (`Hyper-V-VmSwitch Event 102/291`).
2. **Order-7 Allocation Deadlock:** Hyper-V synthetic channel rings (`vmbus_alloc_ring()`) require contiguous physical 512 KiB allocations (`order:7, mode:0xdc0(GFP_KERNEL|__GFP_ZERO)`). Under normal session fragmentation, Order-7 is completely exhausted even when gigabytes of physical memory remain free across orders 0–3, completely wedging session creation.
### Root Cause Forensic Evidence
#### Buddy Allocator State at Failure (/proc/buddyinfo):
```text
Node 0, zone Normal 815 420 120 40 12 8 3 0 0 0 0
```
*(Orders 7–10 are 0; allocator cannot service 512 KiB contiguous blocks).*
#### Kernel Allocation Trace:
```text
kswapd0: page allocation failure: order:0, mode:0x800(GFP_ATOMIC)
Call Trace:
dump_stack_lvl+0x48/0x70
netvsc_alloc_recv_comp+0x28/0x60 [hv_netvsc]
vmbus_onoffer+0x110/0x240 [hv_vmbus]
hv_balloon: balloon inflation requested: 131072 pages (512 MB) from host
hv_balloon: page allocation failure in alloc_balloon_pages (competing with direct reclaim)
```
---
### The Proposed Fixes (Validated on Kernel 6.18.40.1)
#### 1. Dynamic VMBus Atomic Headroom & Balloon Backpressure (`drivers/hv/hv_common.c`, `drivers/hv/hv_balloon.c`)
- Calibrates `vm.min_free_kbytes` dynamically during `late_initcall` (clamped up to 512 MiB) to ensure dedicated reservation for synthetic VMBus channels.
- Defers host balloon inflation (`hv_balloon`) when guest memory is constrained (`si_mem_available() < totalram_pages() / 32`), preventing balloon thrashing during direct reclaim.
#### 2. High-Order Virtual Memory Fallback (`drivers/hv/ring_buffer.c`, `drivers/hv/channel.c`)
- Falls back to `vzalloc()` when physical contiguous order-7 allocation fails.
- Translates virtually allocated pages into PFNs for the GPA descriptor table via `vmalloc_to_page()`, providing 100% transparent operation to the Windows Hyper-V host.
- Fully preserves Confidential VM (Azure CVM / AMD SEV-SNP) memory encryption lifecycle checks (`vfree()` on decrypted pages).
---
### Empirical Hardware Validation
Tested on physical hardware under WSL2 2.7.14.0 (Kernel `6.18.40.1-microsoft-standard-WSL2+`, NVIDIA GeForce RTX 2060):
- **99% RAM Pressure (14.7 GB allocation):** Sustained 100% hold with `PASS_ZERO_PANIC` (Stock WSL2 times out within 45s).
- **Synthetic Channel Latency Under Zero Order-7 Blocks:** Established cleanly in ≤ 0.15 ms via virtual ring buffer.
- **Teardown & Recovery:** 10+ GB clean restored RAM to host upon workload completion.
### Reference Implementation
Complete reference code and build instructions are live on:
👉 **[emersonbusson/WSL2-Linux-Kernel](https://github.com/emersonbusson/WSL2-Linux-Kernel)**
Contributor guide
Research direction
Start by reviewing drivers/hv/hv_common.c, drivers/hv/hv_balloon.c, drivers/hv/ring_buffer.c, and drivers/hv/channel.c, then compare the reference implementation in emersonbusson/WSL2-Linux-Kernel. Done means preventing control-plane allocation failures and balloon thrashing under memory pressure, providing the high-order ring fallback, and preserving teardown and confidential-VM checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, linux
- Domain
- infrastructure, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100