opencontainers / opencontainers/runc
Add integration test to checkpoint / restore container with userns and non default host id
Open
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 13.5k
- Forks
- 2.3k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 30
Description
Description
User namespace is GA since 1.36. However in runc we do not have integration test to check that these containers can be checkpointed / restored (only TestUsernsCheckpoint unit test). I have tried to create simple integration test but i have error on restore.
In criu prepare_cgroup_namespace is called before needs_prep_creds and prepare_userns_creds. However, in prepare_cgns usernsd call is used to move_root.
The child uid and gid are set to send request. However, the kernel returns Invalid argument because nobody uid was passed (check bpftrace log).
If prepare_cgroup_namespace is called after needs_prep_creds and prepare_userns_creds then the test will pass.
Steps to reproduce the issue
- Create integration test
@test "checkpoint userns container with non default host id" {
requires criu root
runc run -d --console-socket "$CONSOLE_SOCKET" test_busybox
[ "$status" -eq 0 ]
testcontainer test_busybox running
runc "$@" checkpoint --work-path ./work-dir test_busybox
[ "$status" -eq 0 ]
testcontainer test_busybox checkpointed
# we need to chown images because child process can try to read them.
chown -R 100000:200000 ./checkpoint
runc "$@" restore -d --work-path ./work-dir --console-socket "$CONSOLE_SOCKET" test_busybox
[ "$status" -eq 0 ]
testcontainer test_busybox running
}
- Check that the latest criu is installed
- Run this integration test
vboxuser@vboxuser:~/runc$ sudo GO=/usr/local/go/bin/go TESTPATH="/userns.bats" make localintegration
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o runc .
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/recvtty
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/sd-helper
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/seccompagent
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/fs-idmap
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/pidfd-kill
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/remap-rootfs
/usr/local/go/bin/go build -trimpath "-buildmode=pie" -tags "seccomp urfave_cli_no_docs" -ldflags "-X main.gitCommit=165a4a02-dirty " -o tests/cmd/_bin ./tests/cmd/key_label
bats -t tests/integration/userns.bats
1..11
ok 1 userns with simple mount
ok 2 userns with 2 inaccessible mounts
ok 3 userns with inaccessible mount + exec
ok 4 userns with bind mount before a cgroupfs mount # skip test requires cgroups_v1
ok 5 userns join other container userns
ok 6 userns join other container userns[selinux enabled] # skip requires SELinux enabled and in enforcing mode
ok 7 userns join other container userns [bind-mounted nsfd]
ok 8 userns join external namespaces [wrong userns owner]
ok 9 userns with network interface
ok 10 userns with network interface renamed
not ok 11 checkpoint userns container with non default host id
# (in test file tests/integration/userns.bats, line 301)
# `[ "$status" -eq 0 ]' failed
# runc spec (status=0)
#
# skipping file /home: cannot remap user 65534:65534 -> -1:-1
# runc run -d --console-socket /tmp/bats-run-11JeBD/runc.baTgfT/tty/sock test_busybox (status=0)
#
# runc state test_busybox (status=0)
# {
# "ociVersion": "1.3.0",
# "id": "test_busybox",
# "pid": 122401,
# "status": "running",
# "bundle": "/tmp/bats-run-11JeBD/runc.baTgfT/bundle",
# "rootfs": "/tmp/bats-run-11JeBD/runc.baTgfT/bundle/rootfs",
# "created": "2026-02-20T12:05:13.911018826Z",
# "owner": ""
# }
# runc checkpoint --work-path ./work-dir test_busybox (status=0)
#
# runc state test_busybox (status=1)
# time="2026-02-20T12:05:14Z" level=error msg="container does not exist"
# runc restore -d --work-path ./work-dir --console-socket /tmp/bats-run-11JeBD/runc.baTgfT/tty/sock test_busybox (status=1)
# time="2026-02-20T12:05:14Z" level=warning msg="--- Quoting \"work-dir/restore.log\""
# time="2026-02-20T12:05:14Z" level=warning msg="293:(00.024142) 1: timens: boottime -1 769220989"
# time="2026-02-20T12:05:14Z" level=warning msg="294:(00.024947) Running setup-namespaces scripts"
# time="2026-02-20T12:05:14Z" level=warning msg="295:(00.024977) \tRPC"
# time="2026-02-20T12:05:14Z" level=warning msg="296:(00.025982) 1: cg: setting cgns prefix to /user.slice/user-1000.slice/test_busybox"
# time="2026-02-20T12:05:14Z" level=warning msg="297:(00.026010) 1: uns: calling userns_move (-1, 0)"
# time="2026-02-20T12:05:14Z" level=warning msg="298:(00.026184) 1: Error (criu/namespaces.c:1417): uns: send req error: Invalid argument"
# time="2026-02-20T12:05:14Z" level=warning msg="299:(00.026221) 1: Error (criu/cgroup.c:1196): cg: couldn't set cgns prefix unified//user.slice/user-1000.slice/test_busybox/cgroup.procs: Invalid argument"
# time="2026-02-20T12:05:14Z" level=warning msg="300:(00.026231) 1: Error (criu/cgroup.c:1287): cg: failed preparing cgns"
# time="2026-02-20T12:05:14Z" level=warning msg="301:(00.031004) uns: calling exit_usernsd (-1, 1)"
# time="2026-02-20T12:05:14Z" level=warning msg="302:(00.031674) uns: daemon calls 0x64c197daa900 (122486, -1, 1)"
# time="2026-02-20T12:05:14Z" level=warning msg="303:(00.031719) uns: `- daemon exits w/ 0"
# time="2026-02-20T12:05:14Z" level=warning msg="304:(00.036754) uns: daemon stopped"
# time="2026-02-20T12:05:14Z" level=warning msg="305:(00.036790) Error (criu/cr-restore.c:2326): Restoring FAILED."
# time="2026-02-20T12:05:14Z" level=warning msg="306:(00.045632) Error (criu/cgroup.c:1998): cg: cgroupd: recv req error: No such file or directory"
# time="2026-02-20T12:05:14Z" level=warning msg=---
# time="2026-02-20T12:05:14Z" level=error msg="criu failed: type RESTORE errno 0"
# --- teardown ---
make: *** [Makefile:174: localintegration] Error 1
Describe the results you received and expected
I have received Criu Restore error. I have expected that there are no errors.
Also i have written simple bpftrace script to check.
root@vboxuser:/home/vboxuser/runc# cat sendmsg.bt
#include <linux/cred.h>
#include <linux/uidgid_types.h>
#include <linux/types.h>
kfunc:vmlinux:__sys_sendmsg /comm == "criu" / {
printf("pid %d tid %d sys send msg is called\n", pid, tid);
}
kretfunc:vmlinux:__sys_sendmsg /comm == "criu" / {
printf("pid %d tid %d sys send msg returned %d\n", pid, tid, retval);
}
kprobe:make_kuid /comm == "criu" / {
$count = ((struct user_namespace *) arg0)->uid_map.nr_extents;
printf("make_kuid is called with namespace mapping count %d\n", $count);
if ($count > 0) {
$mapping = ((struct user_namespace *) arg0)->uid_map.extent[0];
printf("namespace 1 first %d lower_first %d count %d current_uid %u\n", $mapping.first, $mapping.lower_first, $mapping.count, arg1);
}
}
from bpftrace logs:
pid 119540 tid 119540 sys send msg is called
make_kuid is called with namespace mapping count 1
namespace 1 first 0 lower_first 100000 count 65534 current_uid 65534
pid 119540 tid 119540 sys send msg returned -22
from ftrace logs:
1) gdbus-37507 => criu-119540
1) criu-119540 | | __sys_sendmsg() {
1) criu-119540 | 0.392 us | __rcu_read_lock();
1) criu-119540 | 0.342 us | __rcu_read_unlock();
1) criu-119540 | 0.291 us | __rcu_read_lock();
1) criu-119540 | 0.311 us | migrate_disable();
1) criu-119540 | 0.505 us | bpf_get_current_comm();
1) criu-119540 | 0.290 us | bpf_get_current_pid_tgid();
1) criu-119540 | 0.301 us | bpf_get_current_pid_tgid();
1) criu-119540 | | bpf_ringbuf_output() {
1) criu-119540 | | __bpf_ringbuf_reserve() {
1) criu-119540 | 0.298 us | _raw_spin_lock_irqsave();
1) criu-119540 | 0.325 us | _raw_spin_unlock_irqrestore();
1) criu-119540 | 1.893 us | }
1) criu-119540 | 0.298 us | bpf_ringbuf_commit();
1) criu-119540 | 3.145 us | }
1) criu-119540 | 0.310 us | migrate_enable();
1) criu-119540 | 0.284 us | __rcu_read_unlock();
1) criu-119540 | | sockfd_lookup_light() {
1) criu-119540 | 0.606 us | __fdget();
1) criu-119540 | 1.333 us | }
1) criu-119540 | | ___sys_sendmsg() {
1) criu-119540 | | copy_msghdr_from_user() {
1) criu-119540 | 0.312 us | __copy_msghdr();
1) criu-119540 | 1.236 us | }
1) criu-119540 | | ____sys_sendmsg() {
1) criu-119540 | | __check_object_size() {
1) criu-119540 | | __check_object_size.part.0() {
1) criu-119540 | 0.310 us | check_stack_object();
1) criu-119540 | 0.851 us | }
1) criu-119540 | 1.432 us | }
1) criu-119540 | | security_socket_sendmsg() {
1) criu-119540 | | apparmor_socket_sendmsg() {
1) criu-119540 | 0.307 us | aa_unix_msg_perm();
1) criu-119540 | 1.135 us | }
1) criu-119540 | 1.861 us | }
1) criu-119540 | | unix_seqpacket_sendmsg() {
1) criu-119540 | | unix_dgram_sendmsg() {
1) criu-119540 | | wait_for_unix_gc() {
1) criu-119540 | 0.286 us | __cond_resched();
1) criu-119540 | 0.895 us | }
1) criu-119540 | 1.748 us | security_socket_getpeersec_dgram();
1) criu-119540 | | __scm_send() {
1) criu-119540 | | kprobe_ftrace_handler() {
1) criu-119540 | 0.550 us | get_kprobe();
1) criu-119540 | 0.283 us | __rcu_read_lock();
1) criu-119540 | 0.281 us | migrate_disable();
1) criu-119540 | 0.302 us | bpf_get_current_comm();
1) criu-119540 | | copy_from_kernel_nofault() {
1) criu-119540 | 0.284 us | copy_from_kernel_nofault_allowed();
1) criu-119540 | 0.894 us | }
1) criu-119540 | | bpf_ringbuf_output() {
1) criu-119540 | | __bpf_ringbuf_reserve() {
1) criu-119540 | 0.295 us | _raw_spin_lock_irqsave();
1) criu-119540 | 0.297 us | _raw_spin_unlock_irqrestore();
1) criu-119540 | 1.415 us | }
1) criu-119540 | 0.292 us | bpf_ringbuf_commit();
1) criu-119540 | 2.466 us | }
1) criu-119540 | | copy_from_kernel_nofault() {
1) criu-119540 | 0.285 us | copy_from_kernel_nofault_allowed();
1) criu-119540 | 0.797 us | }
1) criu-119540 | | copy_from_kernel_nofault() {
1) criu-119540 | 0.286 us | copy_from_kernel_nofault_allowed();
1) criu-119540 | 0.797 us | }
1) criu-119540 | | copy_from_kernel_nofault() {
1) criu-119540 | 0.285 us | copy_from_kernel_nofault_allowed();
1) criu-119540 | 0.796 us | }
1) criu-119540 | | bpf_ringbuf_output() {
1) criu-119540 | | __bpf_ringbuf_reserve() {
1) criu-119540 | 0.287 us | _raw_spin_lock_irqsave();
1) criu-119540 | 0.283 us | _raw_spin_unlock_irqrestore();
1) criu-119540 | 1.325 us | }
1) criu-119540 | 0.287 us | bpf_ringbuf_commit();
1) criu-119540 | 2.390 us | }
1) criu-119540 | 0.287 us | migrate_enable();
1) criu-119540 | 0.311 us | __rcu_read_unlock();
1) criu-119540 | + 14.631 us | }
1) criu-119540 | | make_kuid() {
1) criu-119540 | 0.316 us | map_id_range_down();
1) criu-119540 | 1.034 us | }
1) criu-119540 | | make_kgid() {
1) criu-119540 | 0.296 us | map_id_range_down();
1) criu-119540 | 0.849 us | }
1) criu-119540 | 0.290 us | put_pid();
1) criu-119540 | + 18.510 us | }
1) criu-119540 | + 22.543 us | }
1) criu-119540 | + 23.242 us | }
1) criu-119540 | + 28.248 us | }
1) criu-119540 | 0.328 us | kfree();
1) criu-119540 | + 31.074 us | }
1) criu-119540 | 0.300 us | __rcu_read_lock();
1) criu-119540 | 0.303 us | migrate_disable();
1) criu-119540 | 0.313 us | bpf_get_current_comm();
1) criu-119540 | 0.287 us | bpf_get_current_pid_tgid();
1) criu-119540 | 0.283 us | bpf_get_current_pid_tgid();
1) criu-119540 | | bpf_ringbuf_output() {
1) criu-119540 | | __bpf_ringbuf_reserve() {
1) criu-119540 | 0.292 us | _raw_spin_lock_irqsave();
1) criu-119540 | 0.297 us | _raw_spin_unlock_irqrestore();
1) criu-119540 | 1.340 us | }
1) criu-119540 | 0.276 us | bpf_ringbuf_commit();
1) criu-119540 | 2.409 us | }
1) criu-119540 | 0.281 us | migrate_enable();
1) criu-119540 | 0.281 us | __rcu_read_unlock();
1) criu-119540 | 0.275 us | __rcu_read_lock();
1) criu-119540 | 0.284 us | __rcu_read_unlock();
1) criu-119540 | + 55.111 us | }
1) criu-119540 => dbus-da-732
What version of runc are you using?
This was checked on current main branch.
vboxuser@vboxuser:~/runc$ git log
commit 165a4a022957d80657e5c716a72cf373bbd7a9d2 (HEAD -> main, origin/main, origin/HEAD)
Merge: f047c6b0 23effed6
Author: Kir Kolyshkin <kolyshkin@gmail.com>
Date: Thu Feb 12 17:34:43 2026 -0800
Merge pull request #5119 from kolyshkin/ubu2404
ci: switch to ubuntu 24.04 for cross-i386 job
vboxuser@vboxuser:~/runc$ criu --version
Version: 4.2
GitID: v4.2-71-gcff99dbcc
Host OS information
vboxuser@vboxuser:~/runc$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.3 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
Host kernel information
vboxuser@vboxuser:~/runc$ uname -a
Linux vboxuser 6.8.0-90-generic #91-Ubuntu SMP PREEMPT_DYNAMIC Tue Nov 18 14:14:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vbox
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/integration/userns.bats and run the named TESTPATH="/userns.bats" make localintegration command with CRIU installed. Review the checkpoint/restore test and the reported CRIU prepare_cgroup_namespace, needs_prep_creds, and prepare_userns_creds ordering. Done means the non-default host-ID userns container checkpoints and restores without errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, go
- Domain
- cli, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100