apple / apple/container

[Request]: Container healthcheck support — configuration, runtime observer, API, and CLI

Open
#1,918 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
49.9k
Forks
1.8k
Avg merge
1d 20h
Merged PRs (30d)
22

Description

### Feature or enhancement request details

## Request

Please add full container healthcheck support to `apple/container`, covering:

1. **Healthcheck specification** — accept a `HealthcheckSpec` on container create (from
the OCI image config or a per-container override), including `HEALTHCHECK NONE` to
disable inherited checks.
2. **Runtime observer** — a daemon-side background task that runs health probes at the
configured cadence, evaluates exit codes, and maintains a `HealthStatus`
(`none | starting | healthy | unhealthy`) with a consecutive-failure streak counter
and a rolling log of recent probe results.
3. **API surface** — expose health state and probe history on `ContainerSnapshot` /
container inspect, and accept healthcheck config at container create time.
4. **CLI surface** — surface health status in `container list` and `container inspect`,
and add a `container healthcheck run` command for on-demand probe invocation.

## Background

A data-shape–only precursor is already proposed in
[#1502](https://github.com/apple/container/issues/1502) /
[#1504](https://github.com/apple/container/pull/1504), which adds the `HealthStatus`
enum and an always-`nil` `health` field on `ContainerSnapshot`. This issue is the
companion full-implementation request, asking for the observer and the richer API/CLI
that actually populate that field.

Docker and Podman have converged on a well-understood healthcheck model (documented
below). Following their design lets tooling authors reuse existing assumptions and
minimises the delta for orchestrators adopting `apple/container`.

## Why this matters

Orchestration tools that run containers depend on health gates, not just liveness
checks:

**Compose spec `depends_on: condition: service_healthy`** — a dependent container must
wait until its dependency transitions to `healthy`. Without a health observer,
orchestrators fall back to treating any running container as healthy, which breaks
workloads that need time to warm up (databases, queue brokers, certificate-dependent
services).

**Microsoft Aspire / DCP** — Aspire's container orchestration layer
([`microsoft/dcp`](https://github.com/microsoft/dcp)) polls container health to decide
when downstream services may start. Lack of healthcheck support is tracked as a
capability gap in [microsoft/dcp#206](https://github.com/microsoft/dcp/issues/206)

**Any health-aware tooling** — service meshes, readiness probes, wait-for scripts, and
CI/CD pipelines that mirror Docker / Podman behaviour expect a `health_status` event
and queryable health state.

## Docker and Podman reference implementation

Both Docker and Podman implement an identical model. The following
is derived from the authoritative Docker Engine API spec (`moby/moby` swagger.yaml).

### Healthcheck configuration (`HealthConfig`)

Used in `ContainerCreate` / OCI image config (`Config.Healthcheck`):

| Field | Type | Default | Notes |
|---|---|---|---|
| `test` | `[String]` | — | `[]` = inherit from image, `["NONE"]` = disable, `["CMD", args…]` = exec directly, `["CMD-SHELL", cmd]` = run via shell |
| `interval` | int64 ns | 30 s | Time between consecutive probes |
| `timeout` | int64 ns | 30 s | Probe is killed with SIGKILL if it exceeds this |
| `retries` | int | 3 | Consecutive failures before transitioning to `unhealthy` |
| `start_period` | int64 ns | 0 s | Grace window after container start; failures here don't increment the streak |
| `start_interval` | int64 ns | 5 s | Probe frequency during `start_period` (Docker Engine 25.0+) |

**Probe exit codes:**
- `0` — healthy
- `1` — unhealthy
- `2` — reserved (treated as unhealthy)
- other — error running probe

### Health state (`Health`)

Returned in `ContainerInspect` / `ContainerState`:

| Field | Type | Notes |
|---|---|---|
| `status` | enum | `none \| starting \| healthy \| unhealthy` |
| `failingStreak` | int | Number of consecutive failures |
| `log` | `[HealthcheckResult]` | Last few results, oldest first |

**`HealthcheckResult`:**

| Field | Type | Notes |
|---|---|---|
| `start` | datetime | RFC 3339 with nanoseconds |
| `end` | datetime | RFC 3339 with nanoseconds |
| `exitCode` | int | 0 = healthy, 1 = unhealthy, other = probe error |
| `output` | string | stdout + stderr from probe (first 4096 bytes stored) |

**State machine:**

```
container starts → status = "starting"
probe passes → status = "healthy"
probe passes during start_period → status = "healthy" (streak resets)
probe fails (streak < retries) → status stays "starting" or "healthy"
probe fails (streak >= retries) → status = "unhealthy"
no HEALTHCHECK in image/config → status = "none"
```

### Container list / summary

`ContainerSummary` (the list endpoint) exposes:
- `health.status` — the summary status enum
- `health.failingStreak` — consecutive failure count

Added in Docker Engine API v1.52; before that, health was only in the full inspect
response.

### Events

Containers emit a `health_status` event whenever the health state transitions. This
lets subscribers react immediately without polling.

### CLI — `podman healthcheck run`

Podman adds a first-class CLI command for on-demand probe invocation:

```
podman healthcheck run [--ignore-result]
```

Exit codes:
- `0` — probe succeeded (healthy)
- `1` — probe failed
- `125` — error (container not running, no healthcheck defined, etc.)

`--ignore-result` exits `0` regardless of the probe result (useful in scripts).

### Dockerfile `HEALTHCHECK` instruction

Images commonly encode healthchecks at build time:

```dockerfile
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD curl -f http://localhost/health || exit 1

# Disable an inherited healthcheck:
HEALTHCHECK NONE
```

The daemon reads `Config.Healthcheck` from the OCI image config automatically.

### CLI — `container create` / `container run` healthcheck flags

Both Docker and Podman expose every `HealthConfig` field as a flag on their `create`
and `run` commands, allowing per-container override of the image-level healthcheck.
`apple/container` should follow the same convention:

| Flag | Maps to | Notes |
|---|---|---|
| `--health-cmd ` | `test` | Shell command to run as the probe; `"none"` disables any healthcheck from the image |
| `--health-interval ` | `interval` | Time between probes, e.g. `30s`, `1m` |
| `--health-timeout ` | `timeout` | Maximum time for a single probe before it is killed |
| `--health-retries ` | `retries` | Consecutive failures before the container is marked `unhealthy` |
| `--health-start-period ` | `startPeriod` | Grace window at startup; failures here don't count against the retry limit |
| `--health-start-interval ` | `startInterval` | Probe frequency during `start-period` |
| `--no-healthcheck` | `test: ["NONE"]` | Disable any healthcheck defined in the image |

All flags are optional. Omitted flags fall back to the value in the OCI image config.
If the image has no healthcheck and no flags are passed, `health` remains `nil` / `none`.

Example:

```sh
container run \
--health-cmd "curl -sf http://localhost/health || exit 1" \
--health-interval 10s \
--health-timeout 3s \
--health-start-period 20s \
--health-retries 3 \
my-image
```

## References

- Docker Engine API v1.55 `HealthConfig` + `Health` schemas:

- Docker `HEALTHCHECK` Dockerfile reference:

- Podman `healthcheck run` man page:

- Data-shape precursor issue: [apple/container#1502](https://github.com/apple/container/issues/1502)
- Data-shape precursor PR: [apple/container#1504](https://github.com/apple/container/pull/1504)
- Microsoft DCP capability gap tracking: [microsoft/dcp#206](https://github.com/microsoft/dcp/issues/206)

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Research direction

No implementation files or tests are named. Start by tracing container creation and the daemon's runtime observer, then inspect ContainerSnapshot, container inspect/list, and the create and healthcheck run CLI entry points. Done means OCI or override configuration is observed, state and probe history are exposed, transitions work, and the documented commands behave as specified.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
backend-api-design, cli, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.