Running docker inside a sysbox-container - inner docker db (containerd snapshotter) gets ruined in case the sysbox-container deleted and re-created
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 3.9k
- Forks
- 230
- Avg merge
- 7h 48m
- Merged PRs (30d)
- 3
Description
I'm running Docker + Sysbox on Ubuntu - running a sysbox-container with an inner Docker engine.
Ubuntu 22.04.1 LTS
Kernel: 5.15.0-43-generic
- Outer
Docker+Sysbox:
# docker --version
Docker version 29.6.1, build 8900f1d
# docker info | grep Storage -A1
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
# sysbox-mgr --version
sysbox-mgr
edition: Community Edition (CE)
version: 0.7.0
commit: bff3721f86e737cfa129dfe9fe2e7933692aba26
built at: Tue Mar 3 21:52:05 UTC 2026
built by: Cesar Talledo
- Inner
Docker:
# docker --version
Docker version 29.5.2, build 79eb04c
# docker info | grep Storage -A1
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
I run the sysbox-container with 2 volumes, so, in case the sysbox-container needs to be deleted and re-created, the inner's docker db will be kept:
docker run --runtime=sysbox-runc ... -v docker_volume:/var/lib/docker -v containerd_volume:/var/lib/containerd ...
After the sysbox-container starts, I can run the following inside fine:
# docker run -it --name alpine_sleep alpine sleep 1
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
55afa1ecc21d: Pull complete
f5124fb579e2: Download complete
56dceff11b33: Download complete
Digest: sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b
Status: Downloaded newer image for alpine:latest
# docker start alpine_sleep
alpine_sleep
Then, I'm deleting and re-creating the sysbox-container and run the commands again.
Note - the new sysbox-container uses the same volumes as before.
# docker start alpine_sleep
Error response from daemon: RWLayer of container deedb7851b87c5e01816fd9b724bd7eb99b93f2ba864a27194f3cb7007a56c2a is unexpectedly nil
failed to start containers: alpine_sleep
# docker image ls
i Info → U In Use
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
alpine:latest 28bd5fe8b56d 3.93MB 3.93MB U
# docker rm alpine_sleep
alpine_sleep
# docker run -it --name alpine_sleep alpine sleep 1
docker: Error response from daemon: NotFound: failed to create snapshot: missing parent "moby/3/sha256:34884abbe92863fce933ed7c39c0e045631af0ed86d5cc0dfbdf9fdca426ce3c" bucket: not found
Run 'docker run --help' for more information
# docker image rm alpine:latest
Untagged: alpine:latest
Deleted: sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b
# docker run -it --name alpine_sleep alpine sleep 1
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
56dceff11b33: Download complete
f5124fb579e2: Download complete
docker: unable to prepare extraction snapshot: AlreadyExists: target snapshot "sha256:34884abbe92863fce933ed7c39c0e045631af0ed86d5cc0dfbdf9fdca426ce3c": already exists
Run 'docker run --help' for more information
As can be seen, some db is corrupted. The container alpine_sleep still exists, but it cannot be started.
Also, the saved image alpine:latest cannot be used.
This flow works fine without --runtime=sysbox-runc (and with --privileged, or other caps instead).
I've tried to --disable-shiftfs, to use file binds instead of volumes. It did not work.
When removing both mounts from the sysbox-container's docker run command: -v ${var_lib_docker_volume}:/var/lib/docker -v ${containerd_volume}:/var/lib/containerd everything works fine. But, I do not have any docker db and need to start everything from scratch.
- Is there a way to make the new
sysbox-containeruse the oldsysbox-container's volumes without an issue? - Are there other files written to the fs which aren't in
/var/lib/dockerand/var/lib/containerdthat I need to put in a volume and share between containers as well?
This seems like a legitimate flow - using persistent volumes. I don't understand why it wouldn't work.
Thank you.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the lifecycle in issue #1021: run an inner Docker engine with --runtime=sysbox-runc, persist /var/lib/docker and /var/lib/containerd, then delete and recreate the sysbox-container. Compare the inner Docker state before and after recreation, focusing on the snapshotter errors and the mounted paths. Done means identifying the missing or mishandled persistent state, or documenting why this volume-reuse flow cannot work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, ubuntu
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100