boxlite-ai / boxlite-ai/boxlite

Multiple directory volume mounts fail with RegisterNetDevice(IrqsExhausted) on Linux/libkrun

Open
#935 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
2.3k
Forks
179
Avg merge
23h 25m
Merged PRs (30d)
121

Description

# Multiple directory volume mounts fail with `RegisterNetDevice(IrqsExhausted)` on Linux/libkrun

## Summary

Using the official BoxLite CLI directly, a box starts successfully with one or two directory volume mounts, but fails to start with three directory volume mounts.

The failure happens before the command runs inside the guest:

```text
Building the microVM failed: RegisterNetDevice(IrqsExhausted)
VM failed to start (libkrun status=-22)
```

This was reproduced without any downstream wrapper. The commands below call the official `boxlite` CLI directly.

## Environment

- BoxLite CLI: `boxlite 0.9.7`
- OS: Ubuntu 22.04.5 LTS
- Kernel: `Linux 5.15.0-40-generic x86_64`
- Architecture: `x86_64`
- CPU: AMD EPYC-Milan Processor
- Hypervisor: KVM
- Test image: `alpine:3.19`

`uname -a`:

```text
Linux OPS-3430 5.15.0-40-generic #43-Ubuntu SMP Wed Jun 15 12:54:21 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
```

## Installation Used for Repro

```bash
mkdir -p /tmp/boxlite-bin
BOXLITE_INSTALL_DIR=/tmp/boxlite-bin curl -fsSL https://sh.boxlite.ai | BOXLITE_INSTALL_DIR=/tmp/boxlite-bin sh
/tmp/boxlite-bin/boxlite --version
```

Output:

```text
boxlite 0.9.7
```

## Reproduction

```bash
ROOT=$(mktemp -d /tmp/boxlite-multi-mount-XXXXXX)

for n in 1 2 3 4 5; do
mkdir -p "$ROOT/dir$n"
echo "dir$n" > "$ROOT/dir$n/file.txt"
done

for count in 2 3; do
echo "=== count=$count ==="

args=()
checks=""
for n in $(seq 1 "$count"); do
args+=("-v" "$ROOT/dir$n:/mnt/dir$n")
checks="$checks cat /mnt/dir$n/file.txt;"
done

BOXLITE_HOME="$ROOT/home-$count" \
BOXLITE_RUNTIME_DIR=/tmp/boxlite-bin/runtime \
/tmp/boxlite-bin/boxlite run --rm "${args[@]}" alpine:3.19 sh -lc "$checks"
done
```

## Expected Behavior

Both runs should start the box and print the mounted file contents:

```text
dir1
dir2
dir3
```

## Actual Behavior

Two directory volumes work:

```text
=== count=2 ===
dir1
dir2
[boxlite] Auto-stopping non-detached box: id=qbrbiLGZcQhr, pid=1775403
```

Three directory volumes fail during VM startup:

```text
=== count=3 ===
Building the microVM failed: RegisterNetDevice(IrqsExhausted)
VM failed to start (libkrun status=-22)
```

Detailed log excerpt:

```text
[shim] T+19ms: entering VM (krun_start_enter)
[krun] 09:11:57.150665 krun_start_enter called
[2026-07-07T09:11:57.233431Z ERROR krun] Building the microVM failed: RegisterNetDevice(IrqsExhausted)
[krun] 09:11:57.238909 krun_start_enter returned (status=-22, elapsed=88ms)
Error: Engine("VM failed to start (libkrun status=-22)")
```

## Additional Check

Running the same three-volume command with `--security disable` still fails with the same error:

```bash
BOXLITE_HOME="$ROOT/home-security-disabled" \
BOXLITE_RUNTIME_DIR=/tmp/boxlite-bin/runtime \
/tmp/boxlite-bin/boxlite run --rm \
--security disable \
-v "$ROOT/dir1:/mnt/dir1" \
-v "$ROOT/dir2:/mnt/dir2" \
-v "$ROOT/dir3:/mnt/dir3" \
alpine:3.19 sh -lc 'cat /mnt/dir1/file.txt; cat /mnt/dir2/file.txt; cat /mnt/dir3/file.txt'
```

Result:

```text
Building the microVM failed: RegisterNetDevice(IrqsExhausted)
VM failed to start (libkrun status=-22)
```

## Notes

From a quick source read, each user `-v/--volume` appears to become a distinct virtiofs share. On the Linux x86_64/libkrun path, there are also internal devices such as shared filesystem, block disks, vsock/network, balloon/rng, etc. The failure may be related to the available IRQ/device limit being exhausted when additional virtiofs devices are added.

The visible failure is reported at net device registration, but the trigger seems to be the increased number of volume-backed virtiofs devices.

Contributor guide

Open the contributing guide

Research direction

Start by running the provided BoxLite CLI reproduction on Linux x86_64, then trace how --volume mounts become virtiofs devices in the libkrun path. Compare the two-mount and three-mount cases, including --security disable; done means three directory mounts start successfully and print dir1, dir2, and dir3.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
cli, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.