Computing digest fails with `No space left on device` despite having space on disk
- Dominant language
- Rust
- Stars
- 2.3k
- Forks
- 230
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 38
Description
Computing digest fails with `No space left on device` despite having space on disk. Empty space on disk is about 10x the image size, nevertheless it fails.
Log:
```
admin@archlinux PIAbootc]$ sudo podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ghcr.io/bootcrew/arch-bootc latest 8e0a3630a6c8 4 hours ago 1.54 GB
localhost/bootc-bin latest 39d5bc27d5a2 19 hours ago 5.23 GB
[admin@archlinux PIAbootc]$ sudo ./hack/build-sealed composefs-sealeduki-sdboot localhost/bootc-bin localhost/bootc --secret=id=secureboot_key,src=target/test-secureboot/db.key --secret=id=secureboot_cert,src=target/test-secureboot/db.crt
DEBUG argv0="bootc"
DEBUG Computing digest of 39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead
DEBUG new_with_config: Spawned skopeo pid=2 config=ImageProxyConfig { authfile: None, auth_data: None, auth_anonymous: false, certificate_directory: None, decryption_keys: None, insecure_skip_tls_verification: None, debug: false, skopeo_cmd: Some(STORAGE_OPTS="additionalimagestore=/run/host-container-storage" "skopeo") }
DEBUG new_with_config: Remote protocol version: 0.2.8 config=ImageProxyConfig { authfile: None, auth_data: None, auth_anonymous: false, certificate_directory: None, decryption_keys: None, insecure_skip_tls_verification: None, debug: false, skopeo_cmd: Some(STORAGE_OPTS="additionalimagestore=/run/host-container-storage" "skopeo") }
DEBUG open_image: opening image self=ImageProxy imgref="containers-storage:39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead"
DEBUG finish_pipe: closing pipe self=ImageProxy pipeid=PipeId(6)
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageConfig, digest: Digest { algorithm: Sha256, value: "sha256:39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead", split: 6 }, size: 10922, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead", split: 6 } size=10922
DEBUG finish_pipe: closing pipe self=ImageProxy pipeid=PipeId(6)
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:242e6849d1986b715e02b4a9025f6457529e3f4470e8bff71388972fa714d8b8", split: 6 }, size: 3684185088, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:242e6849d1986b715e02b4a9025f6457529e3f4470e8bff71388972fa714d8b8", split: 6 } size=3684185088
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:37a50fe28757cfe63f9b025d38d51bf2ecd10710bc862b19e8cffa9731e503c1", split: 6 }, size: 1005191680, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:37a50fe28757cfe63f9b025d38d51bf2ecd10710bc862b19e8cffa9731e503c1", split: 6 } size=1005191680
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:aebb4d4a6be21dce981822641ca56fd82d4d58977403e17040a4e71ca91d0867", split: 6 }, size: 535168512, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:aebb4d4a6be21dce981822641ca56fd82d4d58977403e17040a4e71ca91d0867", split: 6 } size=535168512
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:aa505f34dbdb110b908f01ae6562e5f27a27176d9334d113572530bb419b1a85", split: 6 }, size: 418304, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:aa505f34dbdb110b908f01ae6562e5f27a27176d9334d113572530bb419b1a85", split: 6 } size=418304
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:d4057825841b6856b09b7feadae0a74f98cb7ed209b4c6e017897d50b876232e", split: 6 }, size: 41472, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:d4057825841b6856b09b7feadae0a74f98cb7ed209b4c6e017897d50b876232e", split: 6 } size=41472
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:18d534537d356819a30d2f834c2bfe3b83563081978415efc63ad8f669a9646d", split: 6 }, size: 8704, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:18d534537d356819a30d2f834c2bfe3b83563081978415efc63ad8f669a9646d", split: 6 } size=8704
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:b8d8d24f98190ed3b9a854a8f9d1260893558133f5e16bc67087bd371defb811", split: 6 }, size: 4608, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:b8d8d24f98190ed3b9a854a8f9d1260893558133f5e16bc67087bd371defb811", split: 6 } size=4608
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:8efbe6f58af9febbf6c32f2c35c88cea2eeda70dabed5da56d0668c28e0d84f1", split: 6 }, size: 4096, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:8efbe6f58af9febbf6c32f2c35c88cea2eeda70dabed5da56d0668c28e0d84f1", split: 6 } size=4096
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:918b71a34a40328548dd68ae82c539650673207ba0874ddb5da10bfc50bfe3fe", split: 6 }, size: 1536, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:918b71a34a40328548dd68ae82c539650673207ba0874ddb5da10bfc50bfe3fe", split: 6 } size=1536
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:d65a1579178c68ea1a3cffffa91333528718a838f821ce217c909ccbaabc5ccf", split: 6 }, size: 1536, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:d65a1579178c68ea1a3cffffa91333528718a838f821ce217c909ccbaabc5ccf", split: 6 } size=1536
DEBUG get_descriptor:get_blob: fetching blob self=ImageProxy img=OpenedImage(1) descriptor=Descriptor { media_type: ImageLayer, digest: Digest { algorithm: Sha256, value: "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef", split: 6 }, size: 1024, urls: None, annotations: None, platform: None, artifact_type: None, data: None } self=ImageProxy img=OpenedImage(1) digest=Digest { algorithm: Sha256, value: "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef", split: 6 } size=1024
error: Pulling image: Unable to pull container image containers-storage:39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead: Failed to pull config Descriptor { media_type: ImageConfig, digest: Digest { algorithm: Sha256, value: "sha256:39d5bc27d5a245034df72c68074810eecc9526ee1b5de3fa50d44782a8934ead", split: 6 }, size: 10922, urls: None, annotations: None, platform: None, artifact_type: None, data: None }: No space left on device (os error 28)
[admin@archlinux PIAbootc]$ sudo ./hack/build-sealed composefs-sealeduki-sdboot localhost/bootc-bin localhost/bootc --secret=id=secureboot_key,src=target/test-secureboot/db.key --secret=id=secureboot_cert,src=target/test-secureboot/db.crt^C
[admin@archlinux PIAbootc]$ sudo df -h
Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
dev 7,7G 0 7,7G 0% /dev
run 20G 1,5M 20G 1% /run
efivarfs 256K 159K 93K 64% /sys/firmware/efi/efivars
/dev/vda2 99G 36G 57G 39% /
tmpfs 7,7G 84K 7,7G 1% /dev/shm
tmpfs 1,0M 0 1,0M 0% /run/credentials/systemd-journald.service
tmpfs 7,7G 4,0K 7,7G 1% /tmp
/dev/vda1 1022M 711M 312M 70% /boot
/dev/vda2 99G 36G 57G 39% /home
/dev/vda2 99G 36G 57G 39% /var/cache/pacman/pkg
/dev/vda2 99G 36G 57G 39% /var/log
tmpfs 1,6G 100K 1,6G 1% /run/user/1000
/dev/vdc3 39G 4,9G 32G 14% /mnt/archbootc
/dev/vdc2 1022M 64M 959M 7% /mnt/archbootc/boot
```
Dockerfile:
```
FROM ghcr.io/bootcrew/arch-bootc:latest
RUN rm -rf /opt
RUN mkdir -p /opt
RUN --mount=type=tmpfs,dst=/tmp --mount=type=tmpfs,dst=/var/cache/pacman --mount=type=tmpfs,dst=/usr/lib/sysimage/cache/pacman pacman -Syu --noconfirm && pacman -Scc --noconfirm
RUN --mount=type=tmpfs,dst=/tmp --mount=type=tmpfs,dst=/var/cache/pacman --mount=type=tmpfs,dst=/usr/lib/sysimage/cache/pacman pacman -Sy --noconfirm --needed nano fsverity-utils sddm plasma-meta konsole kate && pacman -Scc --noconfirm
RUN --mount=type=tmpfs,dst=/tmp --mount=type=tmpfs,dst=/var/cache/pacman --mount=type=tmpfs,dst=/usr/lib/sysimage/cache/pacman pacman -Sy whois --noconfirm --needed && pacman -Scc --noconfirm
# RUN usermod -p "$(echo "changeme" | mkpasswd -s)" root
RUN rm -rf /boot /var/cache && \
mkdir /boot /var/cache
RUN bootc container lint
```
If I install less packages, resulting in a smaller image, it works as expected, including further steps like writing to filesystem and booting into it.
Base image Dockerfile: https://github.com/bootcrew/arch-bootc/blob/b2f9a83e2e9c8d7386dbc21db934cc5346771cb6/Containerfile
Contributor guide
Assessment
This issue has not been assessed yet.