containers / containers/podman-compose

up -d hangs indefinitely with depends_on: condition: service_healthy, even when the dependency becomes healthy

Open
#1,541 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.2k
Forks
622
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**

`podman-compose -f compose.yaml up -d` (no `--wait`) hangs indefinitely when a service uses `depends_on: : condition: service_healthy` — confirmed stuck 40+ minutes across two separate CI runs before being cancelled. The hang starts immediately after the dependency's image finishes pulling ("Writing manifest to image destination"), before any container-creation message appears, with zero further output.

The dependency's own healthcheck is not the problem: a separate CI run with the `condition: service_healthy` clause removed entirely shows `postgres` reporting healthy within ~10s. So this isn't the dependency actually failing its healthcheck — `podman-compose`'s own dependency-condition polling appears to hang rather than the underlying health status ever resolving.

**Versions**
- podman-compose: 1.6.0 (installed via `pip install podman-compose` in CI)
- podman: 4.9.3 (Ubuntu-bundled)
- Environment: GitHub Actions `ubuntu-latest` runner

**To Reproduce**

`compose.yaml`:
```yaml
services:
postgres:
image: docker.io/library/postgres:16-alpine@sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777
environment:
POSTGRES_USER: pie
POSTGRES_PASSWORD: pie_password
POSTGRES_DB: pie_db
healthcheck:
test: ["CMD-SHELL", "pg_isready -U pie -d pie_db"]
interval: 5s
timeout: 5s
retries: 10

backend:
image:
depends_on:
postgres:
condition: service_healthy
```

```
podman-compose -f compose.yaml up -d
```

Expected: `backend` starts once `postgres` reports healthy (a few seconds).
Actual: the command hangs indefinitely with no output, well past the point `postgres` would already be healthy.

**What we ruled out**

- **Not a "2 services polling the same condition" issue.** Our real compose file has *two* services depending on `postgres`'s health (`backend` and a second worker service). A first fix attempt removed the condition from only the second service (leaving it depend on `backend` instead), keeping `backend`'s own `postgres: condition: service_healthy` intact. The hang recurred at the identical point — proving a single service alone using the condition is sufficient to trigger it, not concurrent polling from multiple services.
- **Not #1178 / #1183.** Those were fixed by #1184, merged into podman-compose 1.4.0 (2025-05-10). Our CI already runs 1.6.0, which includes that fix — this is a different, still-open issue.
- **Not the dependency's healthcheck config itself** — see above, `postgres` becomes healthy quickly on its own once the condition is removed.

**Workaround**

Removing `condition: service_healthy` entirely (plain `depends_on: [postgres]`, no condition) resolved the hang immediately — verified across multiple CI runs, an isolated local reproduction, and a real production-style release pipeline. We now rely on the app's own crash/restart resilience (`restart: unless-stopped`) plus an external, independent health check (a reverse proxy's active healthcheck) instead of `podman-compose`'s own dependency-condition gate.

Full write-up, reproduction data, and the fix: https://github.com/lautou/pie-manager/pull/71

Contributor guide

Open the contributing guide

Research direction

Start with the compose.yaml reproduction and run `podman-compose -f compose.yaml up -d` using a healthchecked postgres dependency and a long-running backend. Trace the dependency-condition handling after the image pull and compare it with the no-condition workaround. Done means the backend starts after postgres reports healthy and the command returns without hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.