containers / containers/crun

Cannot run any container if storage owned by group not mapped into container with --userns=keep-id

Open
#1,777 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
4.1k
Forks
444
Avg merge
1d 18h
Merged PRs (30d)
43

Description

# Description

This issue is very similar to #1483, if it seems familiar please bear with me, I think there is an important difference. In #1483 the storage folder is inaccessible to the user running podman without their secondary groups, but in this issue it should still be accessible because the user in question owns the folder.

Running a container in rootless mode and `--userns=keep-id` with podman fails if a parent directory of the storage is owned by a group not part of the containers user namespace, and that directory also has no world execute permission.

For example, if the username is `user` with main group `user`, then a directory with permissions `drwx------` owned by `user` and owning group `userdata` as a parent path of the storage location leads to a failure to run any container.

# Steps to reproduce:

1. Use the following `storage.conf` (in `~/.config/containers/`):

```toml
[storage]
driver = "overlay"
graphroot = "/tmp/podman-test/storage"
[storage.options]
mount_program = "/usr/bin/fuse-overlayfs"
```

2. Create a group `userdata` (it's not required to add `user` to this group)
3. Setup the storage folder:

```shell
mkdir /tmp/podman-test
chown user:userdata /tmp/podman-test/
chmod u+rwx /tmp/podman-test/
chmod og-rwx /tmp/podman-test/
```

4. Run a container with a non-root user: `podman run --userns=keep-id alpine ls`

## Expected result:

The container is run successfully.

## Actual result:

The following error is raised:

```plaintext
Error: crun: open `/tmp/graphroot/overlay/5fba3a9a250150294dcb692656d165ec6bd26c9c6be2c692183a70e24083b29c/merged`: Permission denied: OCI permission denied
```

After running `chgrp user /tmp/podman-test/` or `chmod o+x /tmp/podman-test`, the container runs successfully.

# Analysis

Based on me digging around the code and experimenting with strace, I think the diagram below describes what's going on:

Assuming the following user and group ids, and `/etc/subuid`:

```console
> id user
uid=1000(user) gid=1000(user) groups=1000(user)
> grep user /etc/subuid
user:100000:65536
```

Then the simplified view of events is:

```mermaid
sequenceDiagram
participant Podman
participant Crun
participant Kernel

Podman->>Podman: Set up user namespace:
root -> user (Table 1)

Podman->>Crun: start container with:
user -> root (Table 2)
activate Crun

Crun->>Crun: re-invoke in user namespace

Crun->>Kernel: open(root.path)
deactivate Crun
activate Kernel

Note over Kernel: process uid: 0
process gid: 0
file perms: drwx------
file uid:1000
file gid:[invalid]

Kernel--xCrun: return: -1, errno: EPERM
deactivate Kernel
activate Crun
```

`open(root.path)` fails because the kernel requires the uid and gid of the accessed file to be mapped for the root capabilities to take effect [^1]. Normal access checks fail because at this point `crun`s uid is still `0`, so user permissions don't apply.

## Table 1: User Namespace Setup (Podman)
| UID in NS | UID in host |
|--------------|---------|
| [0] | [1000] |
| [1, 65536] | [100000, 165535] |

## Table 2: Container Spec `linux.uidMappings`
| UID in container | UID in outer NS |
|-----------------|----------------|
| [0, 999] | [1, 1000] |
| [1000] | [0] |
| [1001, 65536] | [1001, 65536] |

[^1]: Quoting from [`man 7 user_namespaces`](https://man.archlinux.org/man/user_namespaces.7.en#Operation_of_file-related_capabilities):
> Certain capabilities allow a process to bypass various kernel-enforced restrictions when performing operations on files owned by other users or groups. These capabilities are: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, and CAP_FSETID.
>
> Within a user namespace, these capabilities allow a process to bypass the rules if the process has the relevant capability over the file, meaning that:
> - the process has the relevant effective capability in its user namespace; and
> - **the file's user ID and group ID both have valid mappings in the user namespace.**

(Emphasis mine)

# Possible fixes

1. I think crun could open the storage root path before entering the user namespace, in the same way as its done for mount paths. There the root user in the namespace is still mapped to the user that invoked podman in the host.
2. Or open it after `setuid`.
3. podman could open and pass a file descriptor to crun for the graph storage root.

Option 3 would be the most ideal, because if the user in the host can access the storage root, then IMO containers should be able to start from it, regardless of what permissions enable that access in the host (file ownership, primary or secondary group or ACLs). It's probably not simple to implement however.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.