google / google/gvisor

Add support for multiple image layers in rootfs using custom annotation

Open
#12,348 6 comments 0 reactions 0 assignees View on GitHub
type: enhancement
Dominant language
Go
Stars
19.3k
Forks
2k
Avg merge
3d 5h
Merged PRs (30d)
264

Description

### Description

**Problem Statement**
Currently, runsc only supports specifying a single root filesystem path in config.json When users need layered container images, they must merge these layers (pre-create an overlayfs mount on the host combining all layers) and pass that as a single source. This requires host-level setup and prevents gVisor from natively handling multi-layer container images.

**Proposed Solution**
Add support for a new annotation `dev.gvisor.spec.rootfs.layers` that accepts colon-separated paths to image layers, allowing gVisor to natively handle multi-layer container images.
```
"annotations": {
"dev.gvisor.spec.rootfs.layers": "/base-layer:/app-layer:/config-layer",
"dev.gvisor.spec.rootfs.type": "bind",
"dev.gvisor.spec.rootfs.overlay": "memory"
}
```

**Benefit**
No host kernel mount creation needed

### Is this feature related to a specific bug?

No

### Do you have a specific solution in mind?

The solution leverages gVisor's existing overlayfs support to handle multiple image layers natively:

- Layer Specification: A new annotation dev.gvisor.spec.rootfs.layers accepts colon-separated paths to image layer directories (e.g., /layer1:/layer2:/layer3)

- Gofer Process Setup: A single gofer process handles all layers. Before chroot, the gofer bind-mounts each layer directory to numbered subdirectories (/__layer0, /__layer1, etc.). After chroot, the gofer sees all layers as separate directories within its filesystem

- File Descriptor Creation: For each layer, runsc creates a separate socket pair for LISAFS communication. This results in N file descriptors for N layers. The gofer serves each layer directory over its dedicated socket connection

- Sentry Overlay Creation: Sentry receives N file descriptors (one per layer) for rootfs. For each file descriptor, sentry creates a disconnected LISAFS mount representing that layer. It combines these disconnected mounts into a single overlayfs with multiple read-only lower layers (the image layers) and one writable upper layer (tmpfs). The overlayfs is mounted at the container's root (/)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.