grpc: failed to unmarshal the received message: string field contains invalid UTF-8
Open
Nobody has claimed this yet.
kind/enhancement
priority/p2
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 682
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 29
Description
After upgrading to buildx version v0.19.2, the following error is reported when building the image
[+] Building 3.2s (5/20) docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 829B 0.0s
=> [internal] load metadata for nexus.internal.com/base-image:linux-v2.1.18 0.1s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CANCELED [ 1/16] FROM nexus.internal.com/base-image:linux-v2.1.18@sha256:084539c767645814983a002ca411bfd2a0d80db871d19459f171481fdacf5e3b 2.8s
=> => resolve nexus.internal.com/base-image:linux-v2.1.18@sha256:084539c767645814983a002ca411bfd2a0d80db871d19459f171481fdacf5e3b 0.1s
=> => sha256:084539c767645814983a002ca411bfd2a0d80db871d19459f171481fdacf5e3b 2.85kB / 2.85kB 0.0s
=> => sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc 76.10MB / 76.10MB 1.9s
=> => sha256:777c67696248a1e3d74fb3a1be418d3a393beb8bb362370faa659a282a336006 5.35MB / 5.35MB 0.4s
=> => sha256:d6bfbb7dc215c3a340544c4c87042210e647b140d51a6f210541801fce8274fd 7.78kB / 7.78kB 0.0s
=> => sha256:1ff5115dcffa2e271b232c9dde16c0f7f399773da441c440b497381167582a5f 59.59MB / 142.52MB 2.9s
=> => sha256:11006e65265e13aa0d3f6d56ce11dee5dca992cdcb8a9760431a109aaa79e375 15.73MB / 38.09MB 2.9s
=> => sha256:374999033e8d93a2d24cb2b13d62c8240959ab05c848b8fef1de101a600e7adb 20.97MB / 98.93MB 2.9s
=> => sha256:977e16a77d624fa14338325b88ad9f2aa3059564fa2daac0dce60f8dee15f0dc 2.64MB / 2.64MB 1.1s
=> => sha256:b257af7807da588a476e41081426372189b78526e7c371de2680606c3c02822f 315.64kB / 315.64kB 0.5s
=> => sha256:7e226f4e7f678e9028d1b76c8c0d56d6900058be170ce69baa63ba5567f8f53a 14.68MB / 901.58MB 2.9s
=> => sha256:08be1ee72a529c98177917e0c10bbbc2713519220fad9b3f810b691249a2faa9 193B / 193B 0.5s
=> => sha256:91e577162b472fd8eacc65cf3641be7d736a6d285185d878119e7ae53e4079cd 124B / 124B 0.5s
=> => sha256:8cc69cd3c0537c8a40164bff8c41e624703873f6e31d529d6316ada963d6fa2b 60.82MB / 172.92MB 2.9s
=> => sha256:a941cf7ebf1a67f1c76848a42b49d08eaec0fc9b3d62e4133f2e9d547f1d44f9 15.37MB / 15.37MB 2.4s
=> => extracting sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc 0.9s
=> ERROR [internal] load build context 2.5s
=> => transferring context: 121.49MB 2.4s
------
> [internal] load build context:
------
ERROR: failed to solve: Internal: rpc error: code = Internal desc = grpc: failed to unmarshal the received message: string field contains invalid UTF-8
3057857 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
github.com/moby/buildkit/session/filesync.recvDiffCopy
/root/build-deb/engine/vendor/github.com/moby/buildkit/session/filesync/diffcopy.go:104
github.com/moby/buildkit/session/filesync.FSSync
/root/build-deb/engine/vendor/github.com/moby/buildkit/session/filesync/filesync.go:236
github.com/moby/buildkit/source/local.(*localSourceHandler).snapshot
/root/build-deb/engine/vendor/github.com/moby/buildkit/source/local/source.go:264
github.com/moby/buildkit/source/local.(*localSourceHandler).Snapshot
/root/build-deb/engine/vendor/github.com/moby/buildkit/source/local/source.go:153
github.com/moby/buildkit/solver/llbsolver/ops.(*SourceOp).Exec
/root/build-deb/engine/vendor/github.com/moby/buildkit/solver/llbsolver/ops/source.go:108
github.com/moby/buildkit/solver.(*sharedOp).Exec.func2
/root/build-deb/engine/vendor/github.com/moby/buildkit/solver/jobs.go:1100
github.com/moby/buildkit/util/flightcontrol.(*call[...]).run
/root/build-deb/engine/vendor/github.com/moby/buildkit/util/flightcontrol/flightcontrol.go:122
sync.(*Once).doSlow
/usr/local/go/src/sync/once.go:74
sync.(*Once).Do
/usr/local/go/src/sync/once.go:65
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1695
3066149 v0.19.2 /usr/libexec/docker/cli-plugins/docker-buildx buildx build --debug --network host -t test:test -f ../Dockerfile .
google.golang.org/grpc.(*ClientConn).Invoke
google.golang.org/grpc@v1.66.3/call.go:35
github.com/moby/buildkit/api/services/control.(*controlClient).Solve
github.com/moby/buildkit@v0.18.0/api/services/control/control_grpc.pb.go:88
github.com/moby/buildkit/client.(*Client).solve.func2
github.com/moby/buildkit@v0.18.0/client/solve.go:269
golang.org/x/sync/errgroup.(*Group).Go.func1
golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
runtime.goexit
runtime/asm_amd64.s:1695
3057857 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
github.com/moby/buildkit/solver.(*edge).execOp
/root/build-deb/engine/vendor/github.com/moby/buildkit/solver/edge.go:966
github.com/moby/buildkit/solver/internal/pipe.NewWithFunction[...].func2
/root/build-deb/engine/vendor/github.com/moby/buildkit/solver/internal/pipe/pipe.go:78
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1695
3066149 v0.19.2 /usr/libexec/docker/cli-plugins/docker-buildx buildx build --debug --network host -t test:test -f ../Dockerfile .
github.com/moby/buildkit/client.(*Client).solve.func2
github.com/moby/buildkit@v0.18.0/client/solve.go:285
golang.org/x/sync/errgroup.(*Group).Go.func1
golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
version info:
# docker buildx version
github.com/docker/buildx v0.19.2 1fc5647
# docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default* docker
\_ default \_ default running v0.17.3 linux/amd64 (+3), linux/386
# docker info
Client: Docker Engine - Community
Version: 27.4.0
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.19.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.31.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 6
Running: 6
Paused: 0
Stopped: 0
Images: 6
Server Version: 27.4.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
runc version: v1.2.2-0-g7cb3632
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 5.10.0-26-amd64
Operating System: Debian GNU/Linux 11 (bullseye)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.88GiB
........
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the filesync and local-source paths named in the trace, especially vendor/github.com/moby/buildkit/session/filesync/diffcopy.go:104 and source/local/source.go. Reproduce the build-context transfer with the reported Buildx, Docker, and BuildKit versions, then trace the invalid UTF-8 error across the gRPC call. Done means the reported build no longer fails and the regression is covered by an appropriate test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go, grpc
- Domain
- build-system, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100