containers / containers/fuse-overlayfs

open /var/lib/containers/storage/overlay/.has-mount-program: no such file or directory

Open
#361 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
691
Forks
109
Avg merge
5d 7h
Merged PRs (30d)
2

Description

I am currently working on a GitLab pipeline to build container images. My GitLab runner is running in a docker image in privilege mode and it is using `buildha` to build the below Docker content.
```
FROM registry.access.redhat.com/ubi8/ubi:8.6-855

ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CMD ["/bin/bash"]
```
Extract of `config.toml`
```
[runners.docker]
tls_verify = false
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/var/lib/containers:/var/lib/containers","/cache"]
shm_size = 0
```

In my `.gitlab-ci.yml` I tried using the below command,

```
buildah --storage-opt mount_program=/usr/bin/fuse-overlayfs --storage-driver overlay2 bud -t localhost:test:latest .
```
and I'm getting the below output with an error

```
STEP 1/5: FROM registry.access.redhat.com/ubi8/ubi:8.6-855
Trying to pull registry.access.redhat.com/ubi8/ubi:8.6-855...
Getting image source signatures
Checking if image destination supports signatures
Copying blob sha256:028bdc977650c08fcf7a2bb4a7abefaead71ff8a84a55ed5940b6dbc7e466045
Copying blob sha256:0c673eb68f88b60abc0cba5ef8ddb9c256eaf627bfd49eb7e09a2369bb2e5db0
Copying blob sha256:0c673eb68f88b60abc0cba5ef8ddb9c256eaf627bfd49eb7e09a2369bb2e5db0
Copying blob sha256:028bdc977650c08fcf7a2bb4a7abefaead71ff8a84a55ed5940b6dbc7e466045
Copying config sha256:2fd9e1478809c0c53820517b998e1b2a044207515e5d61eccb6295276bbbaffc
Writing manifest to image destination
Storing signatures
error mounting new container: error mounting build container "373bbfbe192d28bac1ff1cada693eec7c7e9de86db3db07b16e31a79a778ef69": open /var/lib/containers/storage/overlay/.has-mount-program: no such file or directory
```
When I list /var/lib/containers I only see,

```
total 0
drwxr-xr-x. 3 root root 22 Jul 11 04:23 .
drwxr-xr-x. 1 root root 58 Jul 11 04:23 ..
drwxr-xr-x. 2 root root 6 Apr 19 10:19 sigsto
```
Then I added the below command before running `buildha bud` in `.gitlab-ci.yml`,
```
mkdir -p "/var/lib/containers/storage/overlay/" && touch "/var/lib/containers/storage/overlay/.has-mount-program"
```
and managed to move a bit further and got the below error,
```
error committing container for step {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin container=oci] Command:cmd Args:[/bin/bash] Flags:[] Attrs:map[json:true] Message:CMD /bin/bash Original:CMD ["/bin/bash"]}: error copying layers and metadata for container "0c77d5e9339ea2b6dbc82761b7c8b396b2d41c90c36d579b1cdba7fc5d954bba": initializing source containers-storage:ubi-working-container: error extracting layer "8fb7c2252a6873d080ce8a29dc1d4bb1b431ae986a2c19fb0dca0476925f6b8c": stat /var/lib/containers/storage/overlay/8fb7c2252a6873d080ce8a29dc1d4bb1b431ae986a2c19fb0dca0476925f6b8c: no such file or directory
```

If I run with `--storage-driver vfs` I can successfully build the image. @giuseppe can you please shed some light?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the GitLab runner Docker configuration, the buildah storage options, and the mounted /var/lib/containers path shown in the issue. Reproduce the build with the overlay and vfs storage drivers, then inspect how fuse-overlayfs handles the marker and layer paths. Done means identifying why the overlay storage is incomplete and documenting or validating a supported configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, gitlab, linux
Domain
devops, infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.