Dstack-TEE / Dstack-TEE/dstack

Feature: declarative host-service lifecycle for system-container workloads

未关闭
#1,033 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Rust
星标
544
派生
96
平均合并
23 小时 40 分钟
30 天内合并 PR
126

描述

## Summary

We run Incus/LXC system containers inside a DStack CVM. For our storage, cgroup, and reboot semantics, the Incus daemon runs as a PID 1-owned host systemd service rather than as an ordinary application container.

The integration works, but the application currently owns an imperative host-service installer and its complete reboot, readiness, update, and rollback protocol. We would like DStack to provide a constrained, declarative lifecycle contract for this class of host service.

## Existing DStack mechanisms

DStack already provides host execution mechanisms; this request does not claim otherwise.

At current `next` commit `312dc605ebaaf93372534ffb135ea011533d28e9`:

- [`app-compose.service`](https://github.com/Dstack-TEE/dstack/blob/312dc605ebaaf93372534ffb135ea011533d28e9/os/common/rootfs/app-compose.service#L1-L17) is a host systemd oneshot that runs `app-compose.sh` and remains active after the script exits.
- [`app-compose.sh`](https://github.com/Dstack-TEE/dstack/blob/312dc605ebaaf93372534ffb135ea011533d28e9/os/common/rootfs/app-compose.sh#L19-L125) supports a pre-launch script, Docker or nerdctl Compose, and a bash runner.
- The vsock [`HostApi`](https://github.com/Dstack-TEE/dstack/blob/312dc605ebaaf93372534ffb135ea011533d28e9/dstack/host-api/proto/host_api.proto#L30-L34) exposes `Info`, `Notify`, and `GetSealingKey`.

These are useful imperative building blocks, but the current public contract does not describe a typed host-service declaration with persistent/boot-scoped resources, service ownership, readiness, or rollback semantics.

## Why an ordinary Compose service is insufficient for this workload

For our required deployment model, the Incus daemon must:

- own a delegated cgroup subtree for nested system containers;
- use host-native ZFS state under `/dstack/persistent`;
- remain a PID 1-owned service independently of the finite installer; and
- receive explicit persistent and boot-scoped binds into a content-addressed userspace root.

Our current integration therefore uses a finite privileged Compose job with host PID/root access. One centralized `nsenter --target 1` boundary verifies a pinned payload, materializes a boot-scoped root, installs runtime systemd units, binds persistent state and boot-scoped resources, starts the daemon, and checks readiness. A mismatch or partial generation fails closed.

This is workable, but every consumer of this pattern must implement and secure its own installer, convergence, reboot recovery, update, and rollback logic.

## Requested capability

Please consider a constrained host-service declaration that can express:

- a content-addressed and authenticated payload;
- explicit persistent state and boot-scoped runtime mounts;
- bounded systemd unit properties, capabilities, cgroup delegation, and ordering;
- readiness and health conditions; and
- atomic update, reboot recovery, and rollback behavior.

The declaration should be authenticated and capability-scoped. We are not requesting unrestricted host command execution, and the implementation need not extend `HostApi`; a parallel mechanism would be equally suitable.

## Namespace-safe host resources

One concrete edge case is resolver projection. On the released image, `/etc/resolv.conf` was an absolute symlink chain ending under `/run/systemd/resolve`. Reading it through a container's `/host` root bind resolves the absolute target in the container mount namespace and misses the host file. Current mkosi source similarly creates [`/etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf`](https://github.com/Dstack-TEE/dstack/blob/312dc605ebaaf93372534ffb135ea011533d28e9/os/mkosi/mkosi.skeleton/usr/lib/tmpfiles.d/dstack-image.conf#L10-L12).

Our installer handles this inside the PID 1 mount namespace. A declarative contract should instead document or provide a namespace-safe projection for host resources consumed by the declared service. We are not requiring `resolv.conf` to be a regular file.

## Suggested acceptance criteria

- A declared service is restored to ready state after an unchanged CVM reboot.
- Persistent state survives reboot and application update, while boot-scoped resources are reconstructed.
- A failed update leaves or restores the last ready generation.
- The application does not need a host-root bind or `nsenter` to install and reconcile the declared service.

## Scope

This is a feature request, not a report that DStack Compose or `HostApi` is broken. Incus networking/kernel compatibility is tracked separately.

贡献指南

打开贡献指南

调研方向

Read os/common/rootfs/app-compose.service, os/common/rootfs/app-compose.sh, dstack/host-api/proto/host_api.proto, and os/mkosi/mkosi.skeleton/usr/lib/tmpfiles.d/dstack-image.conf to understand the existing host mechanisms and namespace behavior. Define and review the constrained declarative contract against the listed acceptance criteria: reboot restoration, persistent versus boot-scoped resources, atomic rollback, and no application-managed host-root installation.

由索引模型根据 Issue 内容生成。

评估

技术栈
docker, docker-compose, linux, rust
领域
infrastructure, operating-systems, security
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
冷清
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。