containers / containers/crun

Checkpointing of Wasm container with podman+crun fails : `Can't lookup mount`

Open
#1,204 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
4.1k
Forks
444
Avg merge
1d 18h
Merged PRs (30d)
43

Description

This is the same issue than https://github.com/checkpoint-restore/criu/issues/2170. I'm opening it here on the advice of @adrianreber who thinks this issue is related to the Wasm implementation in crun and not a problem within criu.

**Description**

When trying to checkpoint a wasm container started with podman + crun with wasmedge support, the checkpointing fails with an error like:

```
Error (criu/files-reg.c:1710): Can't lookup mount=476 for fd=-3 path=/
Error (criu/cr-dump.c:1524): Collect mappings (pid: 5571) failed with -1
```

This happens on both Fedora 38 (btrfs) and Debian 11 (ext4) up-to-date. For both OSes the error at the end of the `dump.log` file is the same, excepted for the mount number and pid.

**Steps to reproduce the issue:**
1. Create a wasm app. The easiest way is to create a rust app with a simple infinite loop, and compile it for wasm :

```
cargo new app && cd app
rustup target add wasm32-wasi
echo 'fn main() { loop { println!("Hello Wasm");}}' > src/main.rs
cargo build --target wasm32-wasi
```
2. Create the wasm Container from this Containerfile :

```containerfile
FROM scratch
COPY target/wasm32-wasi/debug/app.wasm /app.wasm
CMD ["/app.wasm"]
```

And build with

```
podman build -t demo-wasm --platform wasi/wasm .
```

3. Start this container in the background :

```
podman run --platform wasi/wasm --name demo-wasm-1 -d localhost/demo-wasm
```

You can check it is running with `podman logs demo-wasm-1`. You should see a lot of "Hello Wasm" printed.

4. Try to checkpoint this container with

```
podman container checkpoint demo-wasm-1
```

And notice it is failing.

**Describe the results you received:**
The checkpointing of the container fails

**Describe the results you expected:**
The checkpointing succeeds

**Additional information you deem important (e.g. issue happens only occasionally):**

The issue happens with the most simple of Wasm container. I was able to checkpoint and restore normal containers (debian and others) on the same machine without any issue.

**logs and information:**

Output of `podman container checkpoint` command :

```
2023-05-05T14:21:43.243762Z: CRIU checkpointing failed -52. Please check CRIU logfile /var/lib/containers/storage/overlay-containers/ec5ef8e9db19f3840bfc9357687935de4f7610448552a2be9ab611f2cbd3742e/userdata/dump.log
Error: `/usr/bin/crun-wasm checkpoint --image-path /var/lib/containers/storage/overlay-containers/ec5ef8e9db19f3840bfc9357687935de4f7610448552a2be9ab611f2cbd3742e/userdata/checkpoint --work-path /var/lib/containers/storage/overlay-containers/ec5ef8e9db19f3840bfc9357687935de4f7610448552a2be9ab611f2cbd3742e/userdata ec5ef8e9db19f3840bfc9357687935de4f7610448552a2be9ab611f2cbd3742e` failed: exit status 1
```

`dump.log` file is attached :

[dump.log](https://github.com/containers/crun/files/11407753/dump.log)

Output of `criu --version`:

```
Version: 3.17.1
```

Output of `criu check --all`:

```
Looks good but some kernel features are missing
which, depending on your process tree, may cause
dump or restore failure.
```

Podman version 4.5.0

`crun --version` :

```
crun version 1.8.4
commit: 5a8fa99a5e41facba2eda4af12fa26313918805b
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
```

**Additional environment details:**

Tried on both Fedora 38 (btrfs) and Debian 11 (ext4) in VMs. Criu installed from respective package managers. Outputs are from the Fedora machine. Both crun were using wasmedge as wasm runtime but I'll check if the issue is also present with other wasm runtimes like wasmtime and wasmer.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.