openwrt / openwrt/packages

lxc-start: The container failed to start

Open
#13,037 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Makefile
Stars
4.6k
Forks
4k
Avg merge
3d 12h
Merged PRs (30d)
134

Description

Maintainer: @ratkaj
Environment: OpenWrt Master (as of today) on Raspberry Pi 4B (arm64)
I have made NO modifcations with kernel_menuconfig but I have activated ALL lxc options

Description:
lxc-start does not work:

lxc-start -n compile

lxc-start: compile: lxccontainer.c: wait_on_daemonized_start: 852 Received container state "ABORTING" instead of "RUNNING"
lxc-start: compile: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: compile: tools/lxc_start.c: main: 311 To get more details, run the container in foreground mode
lxc-start: compile: tools/lxc_start.c: main: 314 Additional information can be obtained by setting the --logfile and --logpriority options

This is a fresh devuan container that I have just created with lxc-create -n compile -t download

The debug log shows the following messages:

lxc-start compile 20200805135057.730 WARN     initutils - initutils.c:setproctitle:314 - Invalid argument - Failed to set cmdline
lxc-start compile 20200805135057.730 INFO     lxccontainer - lxccontainer.c:do_lxcapi_start:970 - Failed to set process title to [lxc monitor] /mnt/container/lxc compile
lxc-start compile 20200805135057.731 DEBUG    lxccontainer - lxccontainer.c:wait_on_daemonized_start:830 - First child 1820 exited
lxc-start compile 20200805135057.731 INFO     lsm - lsm/lsm.c:lsm_init:29 - LSM security driver nop
lxc-start compile 20200805135057.731 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:655 - No such device - The process does not have a controlling terminal
lxc-start compile 20200805135057.732 ERROR    cgfsng - cgroups/cgfsng.c:cg_hybrid_get_controllers:656 - Found hierarchy not under /sys/fs/cgroup: "/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpuset,cpu,cpuacct,blkio,pids,rdma
"
lxc-start compile 20200805135057.732 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "reject_force_umount  # comment this to allow umount -f;  not recommended"
lxc-start compile 20200805135057.732 INFO     seccomp - seccomp.c:do_resolve_add_rule:516 - Set seccomp rule to reject force umounts
lxc-start compile 20200805135057.732 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for reject_force_umount action 0(kill)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:do_resolve_add_rule:516 - Set seccomp rule to reject force umounts
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for reject_force_umount action 0(kill)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:do_resolve_add_rule:516 - Set seccomp rule to reject force umounts
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for reject_force_umount action 0(kill)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "[all]"
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "kexec_load errno 1"
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for kexec_load action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for kexec_load action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for kexec_load action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "open_by_handle_at errno 1"
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for open_by_handle_at action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for open_by_handle_at action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for open_by_handle_at action 327681(errno)
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "init_module errno 1"
lxc-start compile 20200805135057.733 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for init_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for init_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for init_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "finit_module errno 1"
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for finit_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for finit_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for finit_module action 327681(errno)
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:770 - Processing "delete_module errno 1"
lxc-start compile 20200805135057.736 INFO     seccomp - seccomp.c:parse_config_v2:967 - Added native rule for arch 0 for delete_module action 327681(errno)
lxc-start compile 20200805135057.737 INFO     seccomp - seccomp.c:parse_config_v2:976 - Added compat rule for arch 1073741864 for delete_module action 327681(errno)
lxc-start compile 20200805135057.737 INFO     seccomp - seccomp.c:parse_config_v2:996 - Added native rule for arch -1073741641 for delete_module action 327681(errno)
lxc-start compile 20200805135057.737 INFO     seccomp - seccomp.c:parse_config_v2:1000 - Merging compat seccomp contexts into main context
lxc-start compile 20200805135057.737 INFO     start - start.c:lxc_init:850 - Container "compile" is initialized
lxc-start compile 20200805135057.737 INFO     cgfsng - cgroups/cgfsng.c:cgfsng_monitor_create:1277 - The monitor process uses "lxc.monitor.compile" as cgroup
lxc-start compile 20200805135057.737 INFO     cgfsng - cgroups/cgfsng.c:cgfsng_payload_create:1356 - The container process uses "lxc.payload.compile" as cgroup
lxc-start compile 20200805135057.738 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWNS
lxc-start compile 20200805135057.738 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWPID
lxc-start compile 20200805135057.738 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWUTS
lxc-start compile 20200805135057.738 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWIPC
lxc-start compile 20200805135057.738 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWNET
lxc-start compile 20200805135057.738 DEBUG    start - start.c:lxc_try_preserve_namespaces:166 - Preserved mnt namespace via fd 18
lxc-start compile 20200805135057.738 DEBUG    start - start.c:lxc_try_preserve_namespaces:166 - Preserved pid namespace via fd 19
lxc-start compile 20200805135057.738 DEBUG    start - start.c:lxc_try_preserve_namespaces:166 - Preserved uts namespace via fd 20
lxc-start compile 20200805135057.738 DEBUG    start - start.c:lxc_try_preserve_namespaces:166 - Preserved ipc namespace via fd 21
lxc-start compile 20200805135057.738 DEBUG    start - start.c:lxc_try_preserve_namespaces:166 - Preserved net namespace via fd 22
lxc-start compile 20200805135057.739 INFO     cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2739 - Limits for the legacy cgroup hierarchies have been setup
lxc-start compile 20200805135057.739 INFO     network - network.c:instantiate_veth:290 - Retrieved mtu 1500 from br-lan
lxc-start compile 20200805135057.762 INFO     network - network.c:instantiate_veth:334 - Attached "vethIJyFrr" to bridge "br-lan"
lxc-start compile 20200805135057.762 DEBUG    network - network.c:instantiate_veth:450 - Instantiated veth tunnel "vethIJyFrr <--> vetha5Cq6Y"
lxc-start compile 20200805135057.762 INFO     start - start.c:do_start:1211 - Unshared CLONE_NEWCGROUP
lxc-start compile 20200805135057.762 DEBUG    storage - storage/storage.c:get_storage_by_name:211 - Detected rootfs type "dir"
lxc-start compile 20200805135057.762 DEBUG    conf - conf.c:lxc_mount_rootfs:1260 - Mounted rootfs "/mnt/container/lxc/compile/rootfs" onto "/usr/lib/lxc/rootfs" with options "(null)"
lxc-start compile 20200805135057.762 INFO     conf - conf.c:setup_utsname:751 - Set hostname to "compile"
lxc-start compile 20200805135057.785 DEBUG    network - network.c:setup_hw_addr:3388 - Mac address "00:11:22:33:44:bc" on "eth0" has been setup
lxc-start compile 20200805135057.816 DEBUG    network - network.c:lxc_network_setup_in_child_namespaces_common:3538 - Network device "eth0" has been setup
lxc-start compile 20200805135057.816 INFO     network - network.c:lxc_setup_network_in_child_namespaces:3560 - Network has been setup
lxc-start compile 20200805135057.816 INFO     conf - conf.c:mount_autodev:1059 - Preparing "/dev"
lxc-start compile 20200805135057.816 DEBUG    conf - conf.c:mount_autodev:1065 - Using mount options: size=500000,mode=755
lxc-start compile 20200805135057.818 INFO     conf - conf.c:mount_autodev:1108 - Prepared "/dev"
lxc-start compile 20200805135057.818 INFO     conf - conf.c:lxc_mount_auto_mounts:680 - Mount source or target for "/usr/lib/lxc/rootfs/proc/sys/net" on "/usr/lib/lxc/rootfs/proc/tty" does not exist. Skipping
lxc-start compile 20200805135057.818 ERROR    utils - utils.c:safe_mount:1148 - No such file or directory - Failed to mount "/usr/lib/lxc/rootfs/proc/tty" onto "/usr/lib/lxc/rootfs/proc/sys/net"
lxc-start compile 20200805135057.818 INFO     conf - conf.c:lxc_mount_auto_mounts:680 - Mount source or target for "/usr/lib/lxc/rootfs/proc/tty" on "/usr/lib/lxc/rootfs/proc/sys/net" does not exist. Skipping
lxc-start compile 20200805135057.818 DEBUG    conf - conf.c:mount_entry:1861 - Remounting "/sys/fs/fuse/connections" on "/usr/lib/lxc/rootfs/sys/fs/fuse/connections" to respect bind or remount options
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:mount_entry:1880 - Flags for "/sys/fs/fuse/connections" were 1070, required extra flags are 14
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:mount_entry:1924 - Mounted "/sys/fs/fuse/connections" on "/usr/lib/lxc/rootfs/sys/fs/fuse/connections" with filesystem type "none"
lxc-start compile 20200805135057.819 INFO     conf - conf.c:lxc_fill_autodev:1152 - Populating "/dev"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/full"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/null"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/random"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/tty"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/urandom"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_fill_autodev:1167 - Created device node "/usr/lib/lxc/rootfs/dev/zero"
lxc-start compile 20200805135057.819 INFO     conf - conf.c:lxc_fill_autodev:1222 - Populated "/dev"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1641 - Created directory for console and tty devices at "/usr/lib/lxc/rootfs/dev/lxc"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1675 - Mounted "/dev/pts/1" onto "/usr/lib/lxc/rootfs/dev/lxc/console"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1681 - Mounted "/dev/pts/1" onto "/usr/lib/lxc/rootfs/dev/lxc/console"
lxc-start compile 20200805135057.819 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1683 - Console has been setup under "/usr/lib/lxc/rootfs/dev/lxc/console" and mounted to "/usr/lib/lxc/rootfs/dev/console"
lxc-start compile 20200805135057.819 INFO     utils - utils.c:lxc_mount_proc_if_needed:1200 - I am 1, /proc/self points to "1"
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_setup_devpts:1521 - Mount new devpts instance with options "gid=5,newinstance,ptmxmode=0666,mode=0620,max=1024"
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_setup_devpts:1536 - Created dummy "/dev/ptmx" file as bind mount target
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_setup_devpts:1541 - Bind mounted "/dev/pts/ptmx" to "/dev/ptmx"
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_allocate_ttys:939 - Created tty "/dev/pts/0" with master fd 16 and slave fd 17
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_allocate_ttys:939 - Created tty "/dev/pts/1" with master fd 18 and slave fd 19
lxc-start compile 20200805135057.821 DEBUG    conf - conf.c:lxc_allocate_ttys:939 - Created tty "/dev/pts/2" with master fd 20 and slave fd 21
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:lxc_allocate_ttys:939 - Created tty "/dev/pts/3" with master fd 22 and slave fd 23
lxc-start compile 20200805135057.822 INFO     conf - conf.c:lxc_allocate_ttys:955 - Finished creating 4 tty devices
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:lxc_setup_ttys:869 - Bind mounted "/dev/pts/0" onto "/dev/lxc/tty1"
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:lxc_setup_ttys:869 - Bind mounted "/dev/pts/1" onto "/dev/lxc/tty2"
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:lxc_setup_ttys:869 - Bind mounted "/dev/pts/2" onto "/dev/lxc/tty3"
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:lxc_setup_ttys:869 - Bind mounted "/dev/pts/3" onto "/dev/lxc/tty4"
lxc-start compile 20200805135057.822 INFO     conf - conf.c:lxc_setup_ttys:900 - Finished setting up 4 /dev/tty<N> device(s)
lxc-start compile 20200805135057.822 INFO     conf - conf.c:setup_personality:1572 - Set personality to "0x0"
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2335 - Dropped mac_admin (33) capability
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2335 - Dropped mac_override (32) capability
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2335 - Dropped sys_time (25) capability
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2335 - Dropped sys_module (16) capability
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2335 - Dropped sys_rawio (17) capability
lxc-start compile 20200805135057.822 DEBUG    conf - conf.c:setup_caps:2338 - Capabilities have been setup
lxc-start compile 20200805135057.822 NOTICE   conf - conf.c:lxc_setup:3433 - The container "compile" is set up
lxc-start compile 20200805135057.822 WARN     cgfsng - cgroups/cgfsng.c:get_hierarchy:171 - There is no useable devices controller
lxc-start compile 20200805135057.822 ERROR    cgfsng - cgroups/cgfsng.c:cg_legacy_set_data:2687 - No such file or directory - Failed to setup limits for the "devices" controller. The controller seems to be unused by "cgfsng" cgroup driver or not enabled on the cgroup hierarchy
lxc-start compile 20200805135057.822 ERROR    cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2731 - No such file or directory - Failed to set "devices.deny" to "a"
lxc-start compile 20200805135057.822 ERROR    start - start.c:lxc_spawn:1787 - Failed to setup legacy device cgroup controller limits
lxc-start compile 20200805135057.822 DEBUG    network - network.c:lxc_delete_network:3693 - Deleted network devices
lxc-start compile 20200805135057.822 ERROR    lxccontainer - lxccontainer.c:wait_on_daemonized_start:852 - Received container state "ABORTING" instead of "RUNNING"
lxc-start compile 20200805135057.822 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start compile 20200805135057.823 ERROR    lxc_start - tools/lxc_start.c:main:311 - To get more details, run the container in foreground mode
lxc-start compile 20200805135057.823 ERROR    lxc_start - tools/lxc_start.c:main:314 - Additional information can be obtained by setting the --logfile and --logpriority options
lxc-start compile 20200805135057.824 ERROR    start - start.c:__lxc_start:1952 - Failed to spawn container "compile"
lxc-start compile 20200805135057.824 WARN     start - start.c:lxc_abort:1025 - No such process - Failed to send SIGKILL via pidfd 17 for process 1822

Is this an bug or am I doing something wrong? I get the message that Cgroup v1 freezer controller is missing with lxc-checkconfig. Can this be the root cause? I have activated "Enable kernel support for LXC" in menuconfig (see screenshot)
Screenshot from 2020-08-05 22-11-32

--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
--- Control groups ---
Cgroups: enabled
Cgroup v1 mount points:
/sys/fs/cgroup
/sys/fs/cgroup/systemd
Cgroup v2 mount points:
Cgroup v1 freezer controller: missing
Cgroup v1 clone_children flag: enabled
Cgroup device: missing
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled, loaded
Macvlan: enabled, loaded
Vlan: enabled, not loaded
Bridges: enabled, not loaded
Advanced netfilter: enabled, not loaded
CONFIG_NF_NAT_IPV4: missing
CONFIG_NF_NAT_IPV6: missing
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: missing
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: missing
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, loaded
FUSE (for use with lxcfs): enabled, loaded
--- Checkpoint/Restore ---
checkpoint restore: missing
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: missing
CONFIG_INET_DIAG: missing
CONFIG_PACKET_DIAG: missing
CONFIG_NETLINK_DIAG: missing
File capabilities:
Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with lxc-start -n compile in foreground mode and capture the logfile using the suggested log options. Read the reported cgfsng cgroup message and the conf.c mount errors in the supplied debug log; done means identifying a reproducible cause and documenting or implementing a verified fix for container startup.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.