microsoft / microsoft/WSL

Custom kernel from linux-msft-wsl-6.18.y branch silently breaks Docker Desktop — no documentation or warning

Open
#40,573 14 comments 0 reactions 1 assignee Claimed by @craigloewen-msft View on GitHub
feature
Dominant language
C++
Stars
33.7k
Forks
1.8k
Avg merge
3d 17h
Merged PRs (30d)
116

Description

# Custom kernel 6.18.y from `linux-msft-wsl-6.18.y` branch breaks Docker Desktop — documentation needed

## Summary

Compiling and using a custom WSL2 kernel from the official [`linux-msft-wsl-6.18.y`](https://github.com/microsoft/WSL2-Linux-Kernel/tree/linux-msft-wsl-6.18.y) branch following the standard procedure (`cp Microsoft/config-wsl .config && make`) results in a kernel that **completely breaks Docker Desktop**. The failure is silent — no useful error in Docker UI, no hint that the kernel is the problem — and requires hours of manual debugging inside the `docker-desktop` distro to diagnose.

There is currently **no documentation** warning users about this incompatibility, nor any guide explaining which kernel options must be built-in (`=y`) or which kernel versions are compatible with Docker Desktop.

## Request

Please provide:

1. **A compatibility notice** on the `linux-msft-wsl-6.18.y` branch README stating that Docker Desktop is not yet compatible with this kernel version
2. **A "Custom Kernel Guide"** documenting:
- Which `CONFIG_*` options MUST be `=y` (not `=m`) for Docker Desktop to function (at minimum: `CONFIG_ISO9660_FS`)
- Which kernel branches are compatible with Docker Desktop (currently only 6.6.y works)
- The fact that `kernel=` in `.wslconfig` is **global** and affects ALL distros including `docker-desktop`
- The fact that WSL2 **never populates `/lib/modules/`**, making any `CONFIG_*=m` effectively dead code
3. **A recommended config fragment** for Docker Desktop compatibility (similar to how Docker provides `check-config.sh` for Linux hosts)

## Environment

| Component | Version |
|-----------|---------|
| Windows | 11 Enterprise 10.0.26200 |
| WSL | 2.7.3.0 |
| Default kernel | 6.6.114.1-microsoft-standard-WSL2 |
| Custom kernel | 6.18.20.3-microsoft-standard-WSL2+ (from `linux-msft-wsl-6.18.y`) |
| Docker Desktop | 4.73.0 (build 226246) |
| Docker Engine | 29.4.3 |

## Problem Description

### What I did

1. Cloned `microsoft/WSL2-Linux-Kernel` branch `linux-msft-wsl-6.18.y`
2. Used the official config: `cp Microsoft/config-wsl .config`
3. Compiled: `make olddefconfig && make -j$(nproc) bzImage`
4. Configured `.wslconfig`:
```ini
[wsl2]
kernel=C:\\wsl-kernels\\6.18.20.3\\bzImage
```
5. Ran `wsl --shutdown` and restarted Docker Desktop

### What happened

Docker Desktop shows "Starting the Docker Engine..." **indefinitely**. The engine never starts. `docker info` returns HTTP 500.

### Debugging (4+ hours)

The failure has **two layers**, both undocumented:

#### Layer 1 — Missing ISO9660 filesystem (silent failure)

The default `Microsoft/config-wsl` sets `CONFIG_ISO9660_FS=m` (module). Since WSL2 never provides `/lib/modules/`, this module can never be loaded. Docker Desktop's `wsl-bootstrap` needs to mount two ISO images during startup:

```
$ wsl -d docker-desktop -u root -- /usr/local/bin/wsl-bootstrap run [...]
mount: unknown filesystem type 'iso9660'
```

**No error appears in Docker Desktop UI.** You must run `wsl-bootstrap` manually to see this.

Evidence:
```bash
$ wsl -d docker-desktop cat /proc/filesystems | grep iso
# (no output)

$ wsl -d docker-desktop zcat /proc/config.gz | grep ISO9660
CONFIG_ISO9660_FS=m

$ wsl -d docker-desktop ls /lib/modules/
ls: No such file or directory
```

#### Layer 2 — Kernel ABI incompatibility (netlink)

After recompiling with `CONFIG_ISO9660_FS=y`, the ISOs mount correctly but Docker's LinuxKit `initd` binary fails:

```
dmesg: netlink: 'initd': attribute type 4 has an invalid length.
logs: dial unix /run/host-services/backend.sock: connect: no such file or directory
```

Docker Desktop's `initd` is compiled against kernel 6.6.x netlink headers. The netlink attribute structures changed in 6.18, causing an ABI mismatch that cannot be fixed by kernel config alone.

### What works

Reverting to the default kernel (commenting out `kernel=` in `.wslconfig`) immediately fixes Docker Desktop:

```bash
$ docker info | grep "Kernel Version"
Kernel Version: 6.6.114.1-microsoft-standard-WSL2
```

## Why this matters

1. **The `linux-msft-wsl-6.18.y` branch exists on the official Microsoft repo** — users reasonably expect it to work with the WSL2 ecosystem including Docker Desktop
2. **The default `Microsoft/config-wsl` is broken for Docker** — `CONFIG_ISO9660_FS=m` is unusable since modules don't work in WSL2
3. **Error messages are completely misleading** — Docker shows "HTTP 500" and "engine not responding", nothing about kernels or filesystems
4. **`kernel=` in `.wslconfig` is a footgun** — it silently affects `docker-desktop` distro with no way to exempt it
5. **Hours of debugging required** — the only way to diagnose this is to manually exec into `docker-desktop`, run `wsl-bootstrap` by hand, and check `dmesg`

## Proposed solutions

### Short term
- Add a **compatibility warning** to the `linux-msft-wsl-6.18.y` README
- Fix `Microsoft/config-wsl` to set `CONFIG_ISO9660_FS=y` (not `=m`) — since modules are dead code in WSL2, there's no reason to use `=m` for anything

### Medium term
- Provide a **"Docker Desktop compatible" config fragment** or a validation script
- Document the `.wslconfig` `kernel=` global scope and its impact on Docker Desktop
- Consider adding a per-distro kernel override mechanism (so `docker-desktop` can use the default kernel while other distros use a custom one)

### Long term
- Coordinate with Docker to ensure LinuxKit binaries support kernel 6.18+ netlink ABI
- Or provide a kernel compatibility layer / stable ABI guarantee for the netlink interface

## Related issues

- Docker for Windows: https://github.com/docker/for-win/issues/15049 (my report)
- Docker feature request for 6.18 support: https://github.com/docker/desktop-feedback/issues/397
- Full debugging procedure and kernel config comparison: https://github.com/valorisa/WSL2-Custom-Kernel-6.18-LTS-Guide/issues/1

## Kernel config comparison (relevant excerpt)

Both kernels were compared (`/proc/config.gz` from default 6.6 vs `.config` from custom 6.18). All Docker-critical options (namespaces, cgroups, netfilter, overlay, virtio, 9p) are **identical**. The only Docker-relevant difference is:

```diff
- CONFIG_ISO9660_FS=m # default 6.6 (works because default kernel has it built differently)
+ CONFIG_ISO9660_FS=y # custom 6.18 (after manual fix)
```

The `initd` failure is NOT a config issue — it's a **kernel ABI incompatibility** between 6.18 netlink structures and Docker's binaries compiled for 6.6.

## Reproduction steps

```powershell
# 1. Compile kernel from linux-msft-wsl-6.18.y using Microsoft/config-wsl
# 2. Set kernel= in .wslconfig pointing to the new bzImage
# 3. wsl --shutdown
# 4. Start Docker Desktop
# 5. Observe infinite "Starting the Docker Engine..." spinner
# 6. Run: docker info → HTTP 500
```

**Expected:** Docker Desktop works, OR a clear error message indicating kernel incompatibility.
**Actual:** Silent infinite hang with misleading HTTP 500 error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.