bake cache misses with multiple targets
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 682
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 29
Description
I was hoping to speed up builds by using the parallelism provided by bake.
Therfore i'm running buildx bake -f docker-bake.hcl
group "default" {
targets = [ "s1","s2","s3","s4","s5","s6","s7","s8","s9","s10","s11","s12","s13","s14","s15","s16","s17","s18","s19","s20","s21"]
}
target "s1" {
context = "./s1"
}
target "s2" {
context = "./s2"
}
target "s3" {
context = "./s3"
}
...
target "s21" {
context = "./s21"
}
but nearly all of the time one (or more) of the images gets rebuilt, even if they should hit the cache.
=> CACHED [s1 build-app.web 2/7] WORKDIR /app 0.0s
=> CACHED [s1 build-app.web 3/7] COPY package*.json /app/ 0.0s
=> CACHED [s1 build-app.web 4/7] RUN npm ci 0.0s
=> CACHED [s1 build-app.web 5/7] COPY ./*.json /app/ 0.0s
=> CACHED [s1 build-app.web 6/7] COPY ./src /app/src 0.0s
=> [s19 build 6/6] RUN npm run build 2.8s
If i specify single targets, the cache is used as i would expect:
for i in {1..21}; do buildx bake -f docker-bake.hcl s$i; done
What if found out so far:
- The more default targets i have, the less likely the cache is used for some of them.
- When i try to reproduce this with trivial Dockerfiles containing only a single
RUNthe problem doesn't occure
Are there any ideas why this is happening or how i could investigate this further?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the supplied docker-bake.hcl and the multi-target buildx bake command, then compare it with the sequential single-target commands. Reduce the example from the 21 targets and contrast it with the trivial Dockerfiles that do not reproduce the problem. Done means identifying why cache use differs for parallel targets and documenting a reliable reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, dockerfile
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100