cncf / cncf/sandbox

[Sandbox] gVisor

Open
#521 1 comment 28 reactions 2 assignees Claimed by @salaboy View on GitHub
contribution-agreement/unsigned New
Dominant language
JavaScript
Stars
197
Forks
40
Avg merge
10h 30m
Merged PRs (30d)
5

Description

### Project summary

gVisor is a sandboxing layer for running containers and agentic workloads efficiently and securely.

### Project description

gVisor is a sandbox that runs workloads securely by **implementing Linux in userspace** in a memory-safe language. We like to quip that **gVisor is the second most mature implementation of Linux, after Linux**.

gVisor does not require virtualization, runs wherever Linux does, and achieves high efficiency in production at scale. gVisor's empirical security track record is on par with that of VM-based sandboxing runtimes.

As opposed to VMs, gVisor offers container-like resource efficiency. Compute resources (CPU, memory) do not need to be explicitly pre-partitioned between the sandbox and the host. Instead, from a resource standpoint, gVisor acts as a normal process that allocates and releases these resources with no additional overhead. This allows it to excel at high-efficiency/high-density workloads. gVisor's kernel boots in milliseconds, supports GPUs and other accelerators, and efficient transparent checkpoint/restore at network speeds. This enables fast scale-out and scale-to-zero use-cases for containerized workloads without compromising security.

Overall, gVisor pairs well with agentic workloads for short LLM-generated code snippet execution, GPU inference workloads with checkpoint/restore, or generic arbitrary code-execution sandbox for defense-in-depth.

gVisor occupies a unique place in the sandboxing ecosystem that **achieves kernel isolation without requiring a separate virtualized kernel**. This involves reimplementation of all Linux subsystems in Go: System calls, process management, memory management, page fault handling, filesystems, ELF loading, networking, signal handling, FUSE, accelerator drivers, etc. As such, it is also often used as a library for other projects that need userspace implementations of these subsystems. In particular, its userspace networking stack is used in many other projects such as Docker Desktop for Mac, WireGuard-Go, Tailscale, and Podman's gvisor-tap-vsock.

### Project vs Reference Architecture/Implementation

- [x] This is a reusable open source project designed for broad adoption, not a reference architecture, reference implementation, or demonstration of patterns

### Org repo URL (provide if all repos under the org are in scope of the application)

N/A

### Project repo URL in scope of application

https://github.com/google/gvisor

### Additional repos in scope of the application

N/A

### Website URL

https://gvisor.dev

### Roadmap

https://gvisor.dev/roadmap

### Roadmap context

The project is already fairly mature and has existed for over a decade, and was open-sourced under an Apache license 8 years ago. As such, its role in the container ecosystem is stable, and its long-term direction is to remain the most secure and efficient application-kernel-based sandbox out there, with improvements to security, application compatibility, performance, efficiency, and ecosystem integration.

### Contributing guide

https://gvisor.dev/contributing

### Code of Conduct (CoC)

https://gvisor.dev/community/code_of_conduct

### Adopters

https://gvisor.dev/users (also https://github.com/google/gvisor/blob/master/ADOPTERS.md)

### Maintainers file

https://github.com/google/gvisor/blob/master/MAINTAINERS.md

### Security policy file

https://gvisor.dev/security / https://github.com/google/gvisor/blob/master/SECURITY.md

### Standard or specification?

gVisor's `runsc` ("**Run** **S**andboxed **C**ontainer") follows the [OCI runtime specification](https://github.com/opencontainers/runtime-spec) and is intended to be drop-in compatible with `runc`.

### Business product or service to project separation

Organizationally, the Google gVisor team is part of the Google Cloud org under Google Kubernetes Engine. However, gVisor is part of the following Google and Alphabet products (non-exhaustive):

- Google Kubernetes Engine (under umbrellas "GKE Sandbox", "GKE Agent Sandbox", and "GKE Pod Snapshots")
- Gemini and Gemini Spark
- Vertex AI
- Google Cloud Run and App Engine
- Google Confidential Space
- YouTube
- BigQuery
- Google Photos
- Gmail
- Google Vulnerability Scanner
- Waymo

As such, while GKE itself has strong dependencies on gVisor, development and usage are already sufficiently decentralized within Google that gVisor _consumption_ and gVisor _development_ are already decoupled, with delineated APIs. As gVisor implements the non-Google-owned OCI runtime specification, integration within other projects can be done permissionlessly and easily, similar to other container ecosystem projects. Within GKE, development operates under an open-source-mostly approach, and this donation will switch gVisor to an open-source-first approach. This brings gVisor development more in line with other software development practices within Google's GKE org (Kubernetes etc.).

There are [other companies that use](https://gvisor.dev/users) and contribute to gVisor who are aiming to co-sponsor the donation.

### Why CNCF?

gVisor doesn't fit neatly into the industry's well-known boxes of the sandboxing/security landscape that tends to separate "vanilla containers" from "virtual machines", with nothing in between. gVisor straddles this middle-ground, providing empirically-equivalent security but without checking the familiar "virtualization" checkbox that security auditors, regulators, or security practitioners often treat as a one-to-one proxy for "secure". This has caused adoption challenges over gVisor's history, as it has been difficult to communicate the value of the project to an audience that is used to this false dichotomy.

Another challenge the project has faced is that of a performance perception problem. Internally within Google (and other gVisor-using companies such as Ant Group and Modal), there exist Linux kernel patches that improve gVisor performance significantly. However, for open-source gVisor users, out-of-the-box performance often shows performance degradation for certain I/O-intensive workloads. This has led to poor first impressions from potential adopters. We have tried to address this by upstreaming Linux kernel patches that improve its performance, but have been turned down by kernel maintainers due to gVisor being perceived as a Google-only project.

Lastly, gVisor as a project has potential that is difficult to prioritize when guided by corporate ownership alone. As a userspace implementation of Linux, gVisor has potential non-commercial applications such as **gVisor-on-Mac** (allowing Linux programs to run natively on OS X without a virtual machine, similar to how Wine allows Windows programs to run natively on Linux without a virtual machine), and **strong sandboxing for desktop Linux** that is much more secure than the current state of the art (bubblewrap/flatpak/nsjail/etc) yet much easier to integrate with than full-blown Qubes-OS-style virtualization-based approaches.

We see evidence of these problems by looking at the current set of gVisor adopters, which are all large tech companies with the ability to investigate and customize gVisor to their own needs (Google, Ant Group, OpenAI, Anthropic), and startups with a highly specific focus that exactly fits gVisor's use-case and where it makes sense to spend a startup's limited resources specifically into making gVisor work great for them (Modal, Tines).
What is missing is hobbyist projects, as well as the "middle" of the industry: individuals and companies that would benefit from gVisor's security, but either aren't aware of its existence, dismiss it out of perception problems, or don't have the resources to invest specifically into security but would happily adopt an off-the-shelf, widely-available sandboxing runtime were it to exist as a widespread and cheap option already available as an offering by their computing infrastructure provider.

By contributing the project to the CNCF, the project aims to address these issues. This enables gVisor and application kernels to become part of the container ecosystem and security industry's lingua franca, enabling integration and adoption beyond highly-motivated/sophisticated/resourceful corporate entities, and enables upstreaming Linux patches that solve gVisor's performance for everyone.

### Benefit to the landscape

> How will adding this project benefit the Cloud Native Landscape?

gVisor provides a missing piece of the container ecosystem by providing security and kernel isolation anywhere Linux runs. Unlike some other security-focused container runtimes, it runs on any x86/ARM hardware, supports GPUs and other accelerators, supports transparent checkpoint/restore, is highly resource-efficient, and empirically does not trade that off for lower security.

> What is the differentiator or enhancement this project provides to existing projects, capabilities [...]?

See the "What CNCF projects does this project complement" section below.

> [...] challenges?

See the previous section for some of the challenges the project faces.
In addition, the project faces competition from other container runtimes, which tend to have better market share and adoption. We attribute most of this popularity difference to a relatively weak marketing muscle on gVisor's end. Beyond that, gVisor always faces criticism for the following attributes:
- **Lower application compatibility**. Because it is a reimplementation of Linux, it necessarily always trails Linux itself, thus it doesn't always implement the latest shiny syscalls. In practice, this has almost never been a problem, but it has still been a _perceptual_ problem by turning off potential adopters because it is impossible to provide a true 100%-confidence "yes, your application will work in gVisor"-type guarantee.
- **Lower _perceived_ security**. gVisor does not share VM-based runtimes' use of virtualization, and there is a perception that this leads to lower security. Empirically, [gVisor's security track record](https://gvisor.dev/security-track-record) is on par with VM-based container runtimes. Within Google's internal security policies, gVisor is treated equivalently as a locked-down VM hypervisor from a security/isolation standpoint.
- **High performance overhead** for some workloads. gVisor's architecture necessitates extra hoops for networking and file-based I/O which can compound into perceptible performance regressions for some workloads. Microbenchmarks, which are adopters' first and easiest way to measure gVisor performance, usually only exercise a very narrow subset of an application's kernel codepaths, and therefore usually either show near-zero or extremely-large performance degradations, neither of which are representative of real-world gVisor use. We strive to continuously improve performance, but full parity is structurally impossible and not a goal. In practice running gVisor in production, we believe the performance problem is 75% perception/25% reality, and that **the compute resource efficiency that gVisor enables** more than makes up for this.

### Cloud native 'fit'

gVisor is a drop-in-compatible container runtime that fits in the Cloud Native container ecosystem. It integrates directly with Cloud Native technologies such as Kubernetes and containerd.

From a resource and efficiency standpoint, gVisor acts as a _more native_ container runtime than other security-focused container runtimes, thanks to its process model. This allows it to be efficiently and tightly sized, enabling container binpacking at a resolution VMs cannot match.

gVisor is already usable on all major clouds, some of which offer it as a native offering, and others for which cloud users can self-install it.

### Cloud native 'integration'

The core gVisor kernel does not depend on any CNCF project. However, to integrate this into cloud-native technologies, gVisor's containerd shim, metrics, and Kubernetes webhook build against containerd, gRPC, and Kubernetes client libraries.

gVisor complements the following CNCF projects:

- [Kubernetes](https://github.com/kubernetes/kubernetes): gVisor is usable as a pod runtime in Kubernetes.
- [containerd](https://github.com/containerd/containerd) and [CRI-O](https://github.com/cri-o/cri-o): gVisor's `runsc` is directly usable as a container runtime under either.
- [Kuasar](https://github.com/kuasar-io/kuasar): gVisor is mentioned as a possible future runtime in Kuasar's README, although support for it does not exist at this time.
- [Falco](https://github.com/falcosecurity/falco): Falco [supports gVisor for threat detection](https://falco.org/blog/intro-gvisor-falco/), although this was deprecated in a later version.
- [Kubernetes Agent Sandbox](https://agent-sandbox.sigs.k8s.io/) has gVisor as a first-class runtime.
- [Prometheus](https://github.com/prometheus/prometheus): gVisor contains a Prometheus-compatible server for monitoring its own health.

### Cloud native overlap

> What CNCF projects does this project overlap with, and how?

gVisor partially overlaps with the following CNCF projects:

- [Hyperlight](https://github.com/hyperlight-dev/hyperlight): gVisor may be used as a library for sandboxing parts of an application. The applications aren't fully-overlapping, as gVisor follows a process model, but gVisor's subsystems can be used as libraries in ways that overlap (e.g. gVisor's userspace network stack).
- [Inclavare Containers](https://github.com/inclavare-containers/inclavare-containers): gVisor is used within Google Confidential Space as a way to isolate the in-enclave Linux kernel from untrusted user workloads, allowing cheap reuse of an existing general-purpose enclave. gVisor essentially provides a higher-security variant of inclavare-containers. In addition, while gVisor does not currently support confidential computing, an implementation of gVisor as a confidential computing runtime is feasible, and would provide a lighter-weight way to run confidential containers with a minimized TCB (as there is no need to run a Linux kernel in the enclave); a prototype of this has been built within Google.
- [Confidential Containers](https://github.com/confidential-containers/confidential-containers): Similar overlap as with Inclavare Containers. gVisor can compose with CoCo as-is by being an in-enclave sandbox within a confidential VM, and could be expanded to a VM-level runtime.
- [urunc](https://github.com/urunc-dev/urunc): Also focused on container security, though the use-cases are non-overlapping. gVisor focuses on Linux-compatible applications and versatility, while urunc requires applications and containers to adopt a unikernel architecture.
- [youki](https://github.com/youki-dev/youki): Also focused on improving container ecosystem security, though its security scope is limited to that of the container runtime binary itself, whereas gVisor provides kernel isolation for the containerized application at runtime.
- [lima](https://github.com/lima-vm/lima): One future potential feature of gVisor we are aiming to catalyze with this donation is the implementation of gVisor-for-Mac, a way to run Linux applications on OS X natively (without a virtual machine), similar to how Wine allows Windows applications to run on Linux natively. This would allow OCI container images and applications to run natively on OS X without the need for Lima's virtual machine.

### Similar projects

> Please list similar projects in the CNCF or elsewhere. If none exist, write "N/A".

gVisor is similar to:

- [Firecracker](https://github.com/firecracker-microvm/firecracker): A MicroVM-based runtime often used in similar contexts gVisor may also be used in. It also focuses on lightweight-ness and containerized workloads.
- [Kata Containers](https://github.com/kata-containers/kata-containers): A MicroVM-based meta-runtime. gVisor is similar to it for the same reason as it is for Firecracker. However, gVisor may also be integrated in Kata Containers as a non-virtual-machine option, by running `kata-agent` inside a gVisor sandbox.
- [Sysbox](https://github.com/nestybox/sysbox): Sysbox virtualizes part of /proc and /sys for a container's view of the filesystem. gVisor fully implements the Linux ABI, providing a strict superset of Sysbox's functionality, as well as superior security.

### Landscape

Yes

### Insights

https://insights.linuxfoundation.org/project/google-gvisor

### Trademark and accounts

- [x] If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

### IP policy

- [x] If the project is accepted, I agree the project will follow the CNCF IP Policy

### Will the project require a license exception?

gVisor is already under Apache 2.0.

### Does the project utilize any dependencies with licenses not on the allowlist or an approved exception?

See [dependencies.yaml](https://github.com/google/gvisor/blob/master/tools/licensecheck/dependencies.yaml) for a list of all gVisor dependencies. All of them fit under existing approved licenses or pre-existing CNCF exceptions, other than one: gVisor's GPU test tooling uses [LLVM](https://github.com/llvm/llvm-project), which is licensed under a slightly-modified Apache 2.0 license. This dependency is used in gVisor's GPU regression test suite, not part of the gVisor release process.

In addition, we have [tooling to audit license dependencies](https://github.com/google/gvisor/tree/master/tools/licensecheck) that ensures no new dependencies can be added without using one of the [CNCF-approved licenses](https://github.com/google/gvisor/blob/master/governance/licensing.yaml).

### Project "Domain Technical Review"

No

### Application contact email(s)

eperot@google.com, stevesilva@google.com, chjing@google.com, smartensson@google.com, bofubo@google.com

### Contributing or sponsoring entity signatory information

| Name | Address | Type (e.g., Delaware corporation) | Signatory name and title | Email address |
|-----------|-----------|-----------|-----------|-----------|
| Srikanth Mandadi | | | Director of Engineering | srimand@google.com |
| Jago Macleod | | | Engineering Director, Kubernetes / GKE | jmacleod@google.com |
| Allan Naim | | | Director, Product Management | anaim@google.com |

### CNCF contacts

Karena Angell (@angellk)

### Additional information

GOVERNANCE.md: https://github.com/google/gvisor/blob/master/GOVERNANCE.md

We are in the process of gathering entities interested in co-sponsoring the donation, and have gotten approval from at least 2 others beyond Google: Ant Group and Modal.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.