docker / docker/buildx

Cant remove cache from pods in external kubernetes context

Open
#870 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/cache area/driver/kubernetes
Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

I don't know is it a bug or more a question. Cant find an answer to this anywhere.

So I created external kubernetes context
docker buildx create --name portal-cluster --driver kubernetes --driver-opt replicas=4,namespace=docker-buildx,loadbalance=sticky
docker buildx use portal-cluster
Build test image
docker buildx build -t nexus.domain.com:445/portal-notary:434 -o type=registry --push --cache-to=type=inline --progress=plain .
Within builder pod I can see that cache created

/var/lib/buildkit # du -sh *
0	buildkitd.lock
52.0K	cache.db
821.9M	runc-overlayfs

If I run buildx build again, as expected it will use some layers from cache.
But I need in some cases prune cache if disk usage become to large.
I tried execute

docker buildx prune --all
docker buildx prune --verbose --builder portal-cluster
docker builder prune --all

But used space inside pod still the same and at build time layers still cached. Only by terminate pod I can delete cache inside pod.
docker buildx du return

Reclaimable:	0B
Total:		0B

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the external Kubernetes builder setup with the shown docker buildx create and build commands, then compare docker buildx prune, docker builder prune, and docker buildx du with the cache under /var/lib/buildkit in the builder pod. Determine why pruning reports 0B while cached layers remain, and verify that the documented or implemented cleanup path actually removes reclaimable cache without terminating the pod.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes
Domain
build-system, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.