docker / docker/buildx

failed to call diskusage: ResourceExhausted

Open
#3,084 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug
Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

Contributing guidelines
I've found a bug and checked that ...
  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem
Description

(me again) hitting gRPC message max sizes, this time it's when running docker buildx du.

cc https://github.com/docker/buildx/issues/2453
cc https://github.com/moby/buildkit/issues/4948
cc https://github.com/moby/moby/issues/48164
cc https://github.com/moby/buildkit/issues/3381 <= probably the same issue

Expected behaviour

No error :)

Actual behaviour
λ docker --debug buildx du
ERROR: failed to call diskusage: ResourceExhausted: grpc: trying to send message larger than max (30520647 vs. 16777216)
280398 v0.22.0 /usr/libexec/docker/cli-plugins/docker-buildx --debug buildx du
google.golang.org/grpc.(*ClientConn).Invoke
	google.golang.org/grpc@v1.69.4/call.go:35
github.com/moby/buildkit/api/services/control.(*controlClient).DiskUsage
	github.com/moby/buildkit@v0.20.1/api/services/control/control_grpc.pb.go:59
github.com/moby/buildkit/client.(*Client).DiskUsage
	github.com/moby/buildkit@v0.20.1/client/diskusage.go:34
github.com/docker/buildx/commands.runDiskUsage.runDiskUsage.func1.func3
	github.com/docker/buildx/commands/diskusage.go:61
golang.org/x/sync/errgroup.(*Group).Go.func1
	golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78
runtime.goexit
	runtime/asm_amd64.s:1700

280398 v0.22.0 /usr/libexec/docker/cli-plugins/docker-buildx --debug buildx du
github.com/moby/buildkit/client.(*Client).DiskUsage
	github.com/moby/buildkit@v0.20.1/client/diskusage.go:36
github.com/docker/buildx/commands.runDiskUsage.runDiskUsage.func1.func3
	github.com/docker/buildx/commands/diskusage.go:61
Buildx version

github.com/docker/buildx v0.22.0 18ccba0

Docker info
Client: Docker Engine - Community
 Version:    28.0.4
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.22.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7
    Path:     /home/pete/.docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 34
 Server Version: 28.0.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-56-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 30.06GiB
 Name: solaralchemist
 ID: 4ff1ee7f-a3ff-4df0-ad6e-9d0162ddbda5
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: fenollp
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Registry Mirrors:
  https://docker-proxy.infra.online.net/
 Live Restore Enabled: false
Builders list
NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker                                 
 \_ default    \_ default       running   v0.20.2    linux/amd64 (+4), linux/386
Configuration

docker buildx du

Build logs

Additional info

Note that I have been building a lot of things with --output type=local that have heavy stage reuse, many contexts and >MB Dockerfiles.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with commands/diskusage.go at the runDiskUsage call and the referenced BuildKit client diskusage.go, then reproduce with docker buildx du using the reported configuration. Done means the command no longer fails with the shown gRPC message-size error for the reported disk-usage workload.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, grpc
Domain
cli, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.