apple / apple/containerization

[Request]: realizing reuse of ext4 layers with overlaybd (a union block device)

Open
#116 10 comments 4 reactions 1 assignee Claimed by @crosbymichael View on GitHub
Dominant language
Swift
Stars
8.9k
Forks
359
Avg merge
2d 20h
Merged PRs (30d)
13

Description

### Feature or enhancement request details

Converting .tgz image layers to block devices + ext4 fs is great for I/O efficiency, but it still has some drawbacks:

* common layers are duplicated in those block devices, resulting waste of storage
* the writable layer is still overlayfs, which is somewhat less performant, and not 100% POSIX-compliant

I suggest that we use overlay block device ([overlaybd](https://github.com/containerd/overlaybd)) as the virtual block device backend, and it:

* consists of separate layer blobs that can be reused by different images
* has built-in writable layer in the form of block device, achieving best performance and 100% POSIX-compliance

As a sub-project of [containerd](https://github.com/containerd/overlaybd), overlaybd is widely used in [Alibaba](https://www.alibabacloud.com/blog/dadi-alibaba-clouds-open-source-accelerated-container-image-technology_597956), [Azure](https://github.com/Azure/peerd/blob/main/docs/usage.md), [DataBricks](https://www.databricks.com/blog/booting-databricks-vms-7x-faster-serverless-compute), [Boss直聘](https://www.zhipin.com/), [Meituan](https://www.meituan.com/), [fly.io](https://community.fly.io/t/experimental-speedy-machine-creation-with-overlaybd/18958), [hocus.dev](https://hocus.dev/blog/qemu-vs-firecracker/), etc. The technology is mature and has been proven in production for years. I believe it will bring unique value to Containerization.

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.