Use annotations instead of labels for Compose-managed container runtime state

Context

Spun off from https://github.com/docker/compose/pull/13959#issuecomment-5296188206 (@thaJeztah):

compose is already heavily depending on labels to store state, but we should start looking if we could transition to using Annotations, especially for state that shouldn't be persisted (doing a docker commit on a container, or running a container from an image that already has labels set could probably still provide "interesting" results)

Problem

Compose stores both identity and runtime state as container labels (com.docker.compose.project, service, container-number, oneoff, config-hash, image, depends_on, replace, version, project.working_dir, project.config_files, environment_file, …). Labels have two persistence problems annotations don't have:

• docker commit on a Compose-managed container bakes all of this state into the resulting image;
• containers inherit image labels, so running a container from such an image (or any image shipping com.docker.compose.* labels) makes it look like Compose-managed state: wrong config-hash comparisons, orphan detection, ps/down matching, etc.

HostConfig.Annotations is per-container, non-identifying runtime metadata — exactly the right home for state that must never leak into images.

Proposal

Split the current label set by role:

• Runtime state → annotations: config-hash, image, depends_on, replace, container-number, oneoff, version, project.working_dir, project.config_files, environment_file — none of these should survive a docker commit.
• Identity stays as labels (for now): project and service are used in server-side filters (docker ps --filter label=…, events, networks/volumes listing); the Engine API has no annotation-based filtering, and annotations don't exist on networks/volumes/images at all. Per the original comment, containerd namespaces may eventually be the right primitive for project scoping — that's engine-side work to track separately.

Backward compatibility

Migration must be invisible to users with running stacks:

1. Phase 1 (dual write, fallback read): new containers get both labels and annotations; all readers prefer the annotation and fall back to the label. Stacks created by older Compose keep working (up recreates, down, ps).
2. Phase 2 (annotation-first): stop writing the state labels on new containers; keep the fallback read path for at least a major-version window.
3. Filters keep using the identity labels throughout; nothing changes for docker ps --filter label=com.docker.compose.project=… consumers, and third-party tooling relying on identity labels is unaffected.

A container recreated by a newer Compose gets the new layout; mixed projects (old + new containers) must reconcile correctly during phase 1, which the fallback read gives us.

Open questions

• Exact split for container-number/oneoff: they are identity-adjacent but not used in server-side filters everywhere — audit each call site.
• Annotations are only available for containers: networks, volumes and images keep labels regardless (their divergence-detection labels like com.docker.compose.network/volume and config-hash stay as-is).
• Whether third-party ecosystems read the state labels (not just identity) — a deprecation note in release notes should cover phase 2.

Known direct label consumers

Compose labels were never a documented API, yet some external tools are known to read com.docker.compose.* labels directly from docker ps output instead of using compose ps or another supported interface — Docker Desktop has been identified as one such consumer. These consumers will need to be updated; the phase 1 dual-write window is their migration period, and does not turn the state labels into a compatibility commitment.

Blocker: the Engine API has no annotation filter

ContainerList returns annotations since Engine v27.0.0 (moby/moby ca0529f984, HostConfig.Annotations in each summary — verified empirically against v29.6.1 / API 1.55), so list-driven reads keep working once state moves to annotations. But server-side filtering by annotation does not exist (invalid filter 'annotation'; label only). This is a blocker for moving any filtered label, and Compose filters on more than identity today:

Consequence: two labels from the runtime state basket are filtered on today. Migrating them requires either an annotation filter landing engine-side, or Compose first rewriting those filter usages — the config-hash presence marker can be replaced by the project label presence, and the container-number equality by client-side selection over a service-scoped list. Both rewrites are Compose-local and should land as a phase 1 prerequisite; identity labels (project, service, oneoff, network) stay labels regardless until the engine offers annotation filtering.

Start by auditing Compose's label writers and readers, especially the ContainerList filtering and lookup paths listed in the issue, then verify the Engine API's annotation support and compatibility constraints. Done means a defined migration plan that preserves identity-label filtering, supports dual-write and fallback reads, and resolves or explicitly tracks the annotation-filter blocker.