containerd / containerd/containerd
[SIG-Node]: KEP-5823: Pod-Level Checkpoint/Restore
- Dominant language
- Go
- Stars
- 21.3k
- Forks
- 4.1k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 92
Description
### KEP/SIG-Node References
- KEP(s): KEP-5823
- stage: alpha
- KEP Issue: https://github.com/kubernetes/enhancements/issues/5823
- KEP PR: https://github.com/kubernetes/enhancements/pull/5851
- K8s-Release: v1.37 (target for Alpha)
- KEP-Owner: @rst0git @viktoriaas @adrianreber @andreyvelich
- SIG-Node member liason: @haircommander
- KEP-Shepherd: TBD
### What is the SIG-Node problem you are trying to solve
KEP-5823 introduces native Pod-level checkpoint and restore in Kubernetes, adding `CheckpointPod` and `RestorePod` RPCs to the CRI `RuntimeService`. These RPCs let the kubelet ask a container runtime to:
- checkpoint a Pod sandbox and the exact set of selected running containers into a runtime-defined checkpoint format;
- restore a new Pod sandbox and its containers from a checkpoint using restore-time sandbox and container configurations; and
- pass optional runtime-handler-specific checkpoint and restore options.
The initial Alpha proposal focus on two use cases: same-node *warm starts* and *fault tolerance*. The source Pod is temporarily frozen while its state is captured, then resumed after the checkpoint attempt. The checkpoint operation itself does not stop, terminate, or delete the source Pod.
The Alpha scope is limited to same-node checkpoint and restore. Cross-node checkpoint transport, live migration, and preservation of device state, volume contents, shared Pod resources, or established TCP connections are deferred to future enhancements.
The CRI API was merged in kubernetes/kubernetes#140366. Without a containerd implementation, kubelet calls to these RPCs return `Unimplemented`, and Kubernetes cannot exercise the KEP's end-to-end checkpoint and restore workflow against containerd.
### Describe the solution you would like
Implement support for `CheckpointPod` and `RestorePod` CRI v1 in containerd's CRI plugin.
The implementation should:
- Update the CRI API dependency to a revision containing the merged Pod checkpoint and restore RPCs, and wire them through the CRI service, instrumentation, clients, and test doubles.
- Keep the checkpoint format private to containerd and the selected sandbox controller/runtime. Kubernetes treats the checkpoint directory as opaque.
- Route checkpoint and restore through an optional sandbox-controller capability so different sandboxers can provide their own implementation and unsupported controllers can return `Unimplemented`.
For `CheckpointPod`:
- Validate the request, caller deadline, output directory, sandbox, and selected container IDs.
- Require every selected container to belong to the requested sandbox and be running, and reject duplicate or incomplete selections.
- Quiesce the selected Pod containers as one coordinated transaction.
- Capture the sandbox configuration, container configurations and statuses, writable filesystem layers, and runtime checkpoint state beneath `output_path`.
- Keep the source Pod running after either success or failure.
- Honor cancellation and gRPC deadlines, remove partial artifacts on failure, and publish only a complete checkpoint.
- Reject unsupported runtime-specific options rather than silently ignoring them.
For `RestorePod`:
- Treat `checkpoint_path` as read-only and validate the checkpoint before changing containerd state.
- Select the requested runtime handler and reject incompatible or unsupported checkpoints.
- Create a new sandbox using the supplied `PodSandboxConfig`.
- Match the supplied `ContainerConfig` entries to checkpointed containers by container name.
- Restore checkpoint-owned process and filesystem state while applying compatible restore-time settings such as mounts, resources, logging paths, labels, annotations, devices, and security constraints.
- Return the new sandbox ID and an exact name-to-container-ID mapping for the restored containers in `CREATED` state so the normal CRI lifecycle can start them.
- Preserve normal containerd runtime selection, metadata stores, cleanup, event, and NRI behavior.
### Additional context
Relevant links:
- WG Checkpoint Restore: https://github.com/kubernetes/community/blob/main/wg-checkpoint-restore
- Meeting notes: https://docs.google.com/document/d/1ZMtHBibXfTw4cQerM4O4DJonzVs3W7Hp2K5ml6pTufs/edit
Contributor guide
Assessment
This issue has not been assessed yet.