devcontainers / devcontainers/images

Rust fails to build projects about half the time on Mac with VirtioFS enabled

Open
#797 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Shell
Stars
2.1k
Forks
963
Avg merge
7h 48m
Merged PRs (30d)
11

Description

When using the [Rust dev container template](https://github.com/devcontainers/templates/tree/main/src/rust) which is customized almost entirely from [this image](https://github.com/devcontainers/images/tree/main/src/rust), a freshly `cargo init`ed project randomly will fail to build about half the time on my basic setup. Disabling VirtioFS lets the builds work reliably.

Supporting VirtioFS is greatly preferred since it presents [significant file system speed improvements on Mac](https://www.docker.com/blog/speed-boost-achievement-unlocked-on-docker-desktop-4-6-for-mac/). It's been generally available (no longer "experimental") [since Docker Desktop 4.15.0](https://www.docker.com/blog/docker-desktop-4-15-improved-usability-and-performance/) (released Dec '22) and is the default [since Docker Desktop 4.23](https://www.docker.com/blog/docker-desktop-4-23/) (Sep '23).

### Reproduction

1. Make sure to use Mac with Docker Desktop
- In my case it's on a MacBook Air 13-inch with M2
- I've recreated with both Docker Desktop 4.23.0 and 4.24.0 (most up-to-date as of writing this)
2. Make sure [VirtioOS is enabled](https://github.com/devcontainers/images/assets/9263640/fea7793f-e3d2-4862-8259-baff6e163669) in your Docker engine
3. Create a new project directory
4. Add the default Rust dev container definition
5. Build and open the project in the dev container
6. Run `cargo init` to initialize a new "hello world" project
7. Repeat the command `rm -fr target && cargo build` notice it fails some of the time
- `rm -fr target` ensures the build happens every time for purposes of this repro

It usually takes about 5 successful builds at most until the first failure happens.

8. Reopen the folder locally
9. Switch to [gRPC FUSE file implementation](https://github.com/devcontainers/images/assets/9263640/583ff84a-5582-454b-9ca7-1bcf52ba2037) in Docker engine settings
10. The window should have disconnected when Docker restarts, reload the window to reconnect
11. Repeat the command `rm -fr target && cargo build`, notice it will no longer fail

I'm not able to find any pattern as to when builds will fail, it seems entirely random.

- I'm not using any special customizations to the container/image
- Docker has plenty of dedicated resources when reproducing this (10 GB RAM and ~100 GB storage)
- It does matter using Rosetta for x86/amd64 emulation, enabling and disabling it doesn't help

### Logs

#### Successful builds

With VirtioFS disabled (gRPC FUSE enabled):

```bash
$ while rm -fr target/ && cargo build; do :; done
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.97s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 1.00s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 1.03s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.98s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 1.00s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.99s
...
```

#### Failed builds

With VirtioFS enabled:

```bash
$ while rm -fr target/ && cargo build; do :; done
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.29s
Compiling example v0.1.0 (/workspaces/example)
Finished dev [unoptimized + debuginfo] target(s) in 0.31s
Compiling example v0.1.0 (/workspaces/example)
error: linking with `cc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/bin:/vscode/vscode-server/bin/linux-arm64/e7e037083ff4455cf320e344325dacb480062c3c/bin/remote-cli:/usr/local/cargo/bin:/usr/local/cargo/bin:/usr/local/cargo/bin:/usr/local/cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/vscode/.local/bin" VSLANG="1033" "cc" "/tmp/rustc0jtGY9/symbols.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.16ymvqt8cxxyt04w.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.1i5hqwsdxzrrmsga.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.29r392z43xk4ppmb.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.2eb8j4q71itcksuj.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.3i82ksilnfvdli6v.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.4gx83nd82i5seyfy.rcgu.o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf.155fsdtb1bfio2qj.rcgu.o" "-Wl,--as-needed" "-L" "/workspaces/example/target/debug/deps" "-L" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-cca993492431c26c.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libpanic_unwind-767683484988fddb.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libobject-33d8ca36e0751c49.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libmemchr-feb6369d9fb8ea6a.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libaddr2line-da5e802c6b261da9.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libgimli-c11c22644b73fbb3.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_demangle-b538894d8056db90.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd_detect-5f75957d64265585.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libhashbrown-eb377e4796f89c63.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-7959bd6dc5a6dc9b.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libminiz_oxide-27e9cf0ad28ebc46.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libadler-9a4d8927e95998a8.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libunwind-7ce72c5f7491b638.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcfg_if-88468c0375a0c64b.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ceea9f1fc4b03431.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liballoc-96085dbefe195fa8.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_core-a8add003da8dc54d.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcore-082d5e9d53946848.rlib" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-21239b3d9ec3b48b.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/usr/local/rustup/toolchains/1.72.1-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "/workspaces/example/target/debug/deps/example-286854bdcea43dbf" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
= note: /usr/bin/ld: cannot find /workspaces/example/target/debug/deps/example-286854bdcea43dbf.1i5hqwsdxzrrmsga.rcgu.o: No such file or directory
collect2: error: ld returned 1 exit status

error: could not compile `example` (bin "example") due to previous error
```

### Specs

Run on my host machine:

```bash
$ docker version
Client:
Cloud integration: v1.0.35+desktop.4
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:28:49 2023
OS/Arch: darwin/arm64
Context: desktop-linux

Server: Docker Desktop 4.23.0 (120376)
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:31:36 2023
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.6.22
GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca
runc:
Version: 1.1.8
GitCommit: v1.1.8-0-g82f18fe
docker-init:
Version: 0.19.0
GitCommit: de40ad0
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.