containers / containers/podman-compose

Multistage builds do not get cached

Open
#1,270 0 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
6.2k
Forks
622
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
If you have a Dockerfile with multiple steps that is being used by multiple services like this:

Repository: https://gitlab.com/chocolateimage/multistage

compose.yaml

```
services:
service1:
build:
target: service1
dockerfile: Dockerfile

service2:
build:
target: service2
dockerfile: Dockerfile

service3:
build:
target: service3
dockerfile: Dockerfile

service4:
build:
target: service4
dockerfile: Dockerfile
```

And the Dockerfile:

```Dockerfile
FROM alpine:latest AS base
WORKDIR /app

FROM base AS deps
COPY dependency-generator.sh /app/
RUN [ "./dependency-generator.sh" ]

FROM base AS service-base
COPY service-runner.sh /app

# Pretend all of the services below do different things

FROM service-base AS service1
COPY --from=deps /app/deps /app/deps
ENTRYPOINT [ "./service-runner.sh", "service1" ]

FROM service-base AS service2
COPY --from=deps /app/deps /app/deps
ENTRYPOINT [ "./service-runner.sh", "service2" ]

FROM service-base AS service3
COPY --from=deps /app/deps /app/deps
ENTRYPOINT [ "./service-runner.sh", "service3" ]

FROM service-base AS service4
COPY --from=deps /app/deps /app/deps
ENTRYPOINT [ "./service-runner.sh", "service4" ]
```

The `deps` stage will be executed multiple times, this does not happen with Docker itself. In this example it will not do much, but in a real setting where you would download stuff and do CPU intensive work, this can slow down the build process by a large amount.

**To Reproduce**
Steps to reproduce the behavior:
1. `git clone https://gitlab.com/chocolateimage/multistage.git`
2. `podman-compose build`
3. The line `"Generating" your dependencies` gets printed multiple times

**Expected behavior**
The stage for generating dependencies only gets executed once and cached for the other `COPY --from` commands, just like in Docker.

**Actual behavior**
What is the behavior you actually got and that should not happen.

**Output**

```
$ podman-compose version
podman-compose version 1.5.0
podman version 5.5.2

$ podman version
Client: Podman Engine
Version: 5.5.2
API Version: 5.5.2
Go Version: go1.24.4
Git Commit: e7d8226745ba07a64b7176a7f128e4ef53225a0e
Built: Fri Jun 27 10:15:44 2025
OS/Arch: linux/amd64

$ podman-compose build
[1/7] STEP 1/2: FROM alpine:latest AS base
[1/4] STEP 1/2: FROM alpine:latest AS base
[1/5] STEP 1/2: FROM alpine:latest AS base
[1/6] STEP 1/2: FROM alpine:latest AS base
[1/4] STEP 2/2: WORKDIR /app
[1/5] STEP 2/2: WORKDIR /app
[1/6] STEP 2/2: WORKDIR /app
--> Using cache 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7
--> 95933deb20a3
--> Using cache 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7
[3/4] STEP 1/2: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS service-base
--> 95933deb20a3
[3/5] STEP 1/2: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS service-base
[1/7] STEP 2/2: WORKDIR /app
--> Using cache 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7
--> 95933deb20a3
[3/4] STEP 2/2: COPY service-runner.sh /app
[3/5] STEP 2/2: COPY service-runner.sh /app
[2/6] STEP 1/3: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS deps
[2/6] STEP 2/3: COPY dependency-generator.sh /app/
--> Using cache 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7
--> 95933deb20a3
[2/7] STEP 1/3: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS deps
[2/7] STEP 2/3: COPY dependency-generator.sh /app/
--> Using cache 034fcc266fb6114d2937ab288fa83ea807d0e66a8feb3dc82ad0c98314fd540d
--> 034fcc266fb6
[2/6] STEP 3/3: RUN [ "./dependency-generator.sh" ]
--> Using cache 034fcc266fb6114d2937ab288fa83ea807d0e66a8feb3dc82ad0c98314fd540d
--> 034fcc266fb6
[2/7] STEP 3/3: RUN [ "./dependency-generator.sh" ]
"Generating" your dependencies...
--> a56b1948e08e
[4/4] STEP 1/3: FROM a56b1948e08e85a3b8f4afe5b1f38ae741e65042e3de3bb0ac4f1fd4665232ca AS service1
[4/4] STEP 2/3: COPY --from=deps /app/deps /app/deps
[2/4] STEP 1/3: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS deps
"Generating" your dependencies...
[2/4] STEP 2/3: COPY dependency-generator.sh /app/
--> e0bd0f2bd1b2
[2/5] STEP 1/3: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS deps
[2/5] STEP 2/3: COPY dependency-generator.sh /app/
--> Using cache 034fcc266fb6114d2937ab288fa83ea807d0e66a8feb3dc82ad0c98314fd540d
--> 034fcc266fb6
[2/4] STEP 3/3: RUN [ "./dependency-generator.sh" ]
--> Using cache 034fcc266fb6114d2937ab288fa83ea807d0e66a8feb3dc82ad0c98314fd540d
--> 034fcc266fb6
[2/5] STEP 3/3: RUN [ "./dependency-generator.sh" ]
"Generating" your dependencies...
"Generating" your dependencies...
Done
Done
--> 385e7c36c476
--> dd616ccca2ac
Done
[3/7] STEP 1/2: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS service-base
--> Using cache 385e7c36c476103bb79d835758d030cf0deb5ac2a42783f480b2d27d99c694dd
--> 385e7c36c476
[3/6] STEP 1/2: FROM 95933deb20a39176801eb47bad90d1eebb5a975555ad3cc92231bdc5efec4de7 AS service-base
[3/7] STEP 2/2: COPY service-runner.sh /app
[3/6] STEP 2/2: COPY service-runner.sh /app
Done
--> Using cache 385e7c36c476103bb79d835758d030cf0deb5ac2a42783f480b2d27d99c694dd
--> 385e7c36c476
[5/5] STEP 1/3: FROM e0bd0f2bd1b2f166f2240ada9817708cc89e344e0c06d67c082fc59d82fa3a09 AS service2
[5/5] STEP 2/3: COPY --from=deps /app/deps /app/deps
--> Using cache a56b1948e08e85a3b8f4afe5b1f38ae741e65042e3de3bb0ac4f1fd4665232ca
--> a56b1948e08e
--> Using cache a56b1948e08e85a3b8f4afe5b1f38ae741e65042e3de3bb0ac4f1fd4665232ca
--> a56b1948e08e
[6/6] STEP 1/3: FROM a56b1948e08e85a3b8f4afe5b1f38ae741e65042e3de3bb0ac4f1fd4665232ca AS service3
[7/7] STEP 1/3: FROM a56b1948e08e85a3b8f4afe5b1f38ae741e65042e3de3bb0ac4f1fd4665232ca AS service4
[6/6] STEP 2/3: COPY --from=deps /app/deps /app/deps
[7/7] STEP 2/3: COPY --from=deps /app/deps /app/deps
--> 7ae956fbd264
--> 15ca8285fd4c
--> a9360b9c3962
--> 0245a875c6f2
[5/5] STEP 3/3: ENTRYPOINT [ "./service-runner.sh", "service2" ]
[4/4] STEP 3/3: ENTRYPOINT [ "./service-runner.sh", "service1" ]
[7/7] STEP 3/3: ENTRYPOINT [ "./service-runner.sh", "service4" ]
[6/6] STEP 3/3: ENTRYPOINT [ "./service-runner.sh", "service3" ]
[5/5] COMMIT multistage_service2
[6/6] COMMIT multistage_service3
[4/4] COMMIT multistage_service1
[7/7] COMMIT multistage_service4
--> 7af36609ab82
Successfully tagged localhost/multistage_service4:latest
--> 74767c5e34ff
--> f257e791174d
Successfully tagged localhost/multistage_service3:latest
Successfully tagged localhost/multistage_service2:latest
--> 108129ec59fd
Successfully tagged localhost/multistage_service1:latest
7af36609ab8204af22d5df7bdde61fbf9320a61eec2dbf17df9e3cca16b5e997
74767c5e34ffd950a0da07fbb84e788e722d90fdc389b5403d7c935a5c720ab4
f257e791174d390653459aeafc5502c929e13558c035e7ca7fe2fee37ea028c7
108129ec59fdac8160f06ae07fa8baba7726782d616fd46aa8afd86df3232705

```

**Environment:**
- OS: Linux
- podman version: 5.5.2
- podman compose version: f7eeda1a3db10952424af6a5b0501c269ebe3f0d

**Additional context**

Tested in a fresh libvirt/QEMU machine with Arch Linux. I uploaded the example code on GitLab: https://gitlab.com/chocolateimage/multistage

Contributor guide

Open the contributing guide

Research direction

Start by cloning the linked multistage repository and running podman-compose build to reproduce the repeated dependency generation. Then trace how podman-compose handles multiple build targets and shared multistage stages; done means the deps stage runs once and its result is reused while all four services build successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, python
Domain
build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
39/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.