Rust-GPU / Rust-GPU/rust-gpu

Stack overflow during shader compilation

Open
#461 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
3.4k
Forks
125
PR merge metrics
No merged PRs in 30d

Description

Expected Behaviour

Here is an issue I just faced when replacing a loop with two direct calls:

error: failed to run custom build command for `ab-proof-of-space-gpu v0.1.0 (/web/github/abundance/crates/farmer/ab-proof-of-space-gpu)`

Caused by:
  process didn't exit successfully: `/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-62f09b5708f23384/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=RUSTGPU_CODEGEN_ARGS
  cargo:rerun-if-env-changed=RUSTGPU_RUSTFLAGS
  cargo:rerun-if-env-changed=RUSTGPU_CARGOFLAGS

  --- stderr
     Compiling ab-proof-of-space-gpu v0.1.0 (/web/github/abundance/crates/farmer/ab-proof-of-space-gpu)
  error: rustc interrupted by SIGSEGV, printing backtrace

  /home/nazar-pc/.rustup/toolchains/nightly-2025-06-30-x86_64-unknown-linux-gnu/lib/librustc_driver-d3e8ca52c6974f9d.so(+0x3ba5b2f)[0x7fe0de5a5b2f]
  /lib/x86_64-linux-gnu/libc.so.6(+0x45330)[0x7fe0da645330]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0x104223f)[0x7fe0ce04223f]

  ### cycle encountered after 3 frames with period 6
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0xff3f85)[0x7fe0cdff3f85]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0x100f353)[0x7fe0ce00f353]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0x10424c9)[0x7fe0ce0424c9]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0xff3f85)[0x7fe0cdff3f85]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0x100f353)[0x7fe0ce00f353]
  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0x10424c9)[0x7fe0ce0424c9]
  ### recursed 42 times

  /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so(+0xff3f85)[0x7fe0cdff3f85]

  note: rustc unexpectedly overflowed its stack! this is a bug
  note: maximum backtrace depth reached, frames may have been lost
  note: we would appreciate a report at https://github.com/rust-lang/rust
  help: you can increase rustc's stack size by setting RUST_MIN_STACK=16777216
  error: could not compile `ab-proof-of-space-gpu` (lib)

  Caused by:
    process didn't exit successfully: `/home/nazar-pc/.rustup/toolchains/nightly-2025-06-30-x86_64-unknown-linux-gnu/bin/rustc --crate-name ab_proof_of_space_gpu --edition=2024 crates/farmer/ab-proof-of-space-gpu/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type dylib --crate-type lib --emit=dep-info,link -C embed-bitcode=no -C codegen-units=1 -C debuginfo=2 --warn=rust_2018_idioms --warn=missing_debug_implementations --allow=unexpected_cfgs --check-cfg 'cfg(target_arch, values("spirv"))' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=420c5aaec24189dd --out-dir /home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps --target /home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/target-specs/spirv-unknown-vulkan1.2.json -C incremental=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/incremental -L dependency=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps -L dependency=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/debug/deps --extern ab_blake3=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libab_blake3-a4e221411a36f2cf.rlib --extern ab_chacha8=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libab_chacha8-13720534e6d2d2e7.rlib --extern 'noprelude:compiler_builtins=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libcompiler_builtins-372bb16695370e45.rlib' --extern 'noprelude:core=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libcore-876111756530f1ee.rlib' --extern derive_more=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libderive_more-76a08933c84dae4d.rlib --extern spirv_std=/home/nazar-pc/.cache/cargo/target/debug/build/ab-proof-of-space-gpu-fecff1fdb1528c5b/out/spirv-unknown-vulkan1.2/debug/deps/libspirv_std-d8b5ef86ea70b0dd.rlib -Z unstable-options -Zcodegen-backend=/home/nazar-pc/.cache/rust-gpu/codegen/https___github_com_Rust-GPU_rust-gpu+6a9ea3bb/librustc_codegen_spirv.so -Zbinary-dep-depinfo -Csymbol-mangling-version=v0 '-Zcrate-attr=feature(register_tool)' '-Zcrate-attr=register_tool(rust_gpu)' -Coverflow-checks=off -Cdebug-assertions=off -Zinline-mir=off -Zmir-enable-passes=-GVN -Zshare-generics=off -Cllvm-args=--spirv-metadata=name-variables -Ctarget-feature=+GroupNonUniformArithmetic,+GroupNonUniformBallot,+GroupNonUniformShuffle` (signal: 11, SIGSEGV: invalid memory reference)

Looks like recursion that went too deep somewhere.

RUST_MIN_STACK=16777216 suggestion did help, so it is not an infinite loop or anything like that, just uses too much stack space due to recursion.

Example & Steps To Reproduce

Remove this line: https://github.com/nazar-pc/abundance/blob/1639a5172a9792d4989a4615de264f23106752be/crates/farmer/ab-proof-of-space-gpu/build.rs#L38

Then:

cd crates/farmer/ab-proof-of-space-gpu
cargo build

System Info

Rust-gpu from 6a9ea3bb96d34b2d5211a34d09fae81b0c938916 and cargo-gpu from https://github.com/Rust-GPU/cargo-gpu/commit/09829681cbe3866b6d1fc2ee0eae83aa77855b3e

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/farmer/ab-proof-of-space-gpu/build.rs at the referenced line and reproduce with cd crates/farmer/ab-proof-of-space-gpu && cargo build after removing it. Inspect the rust-gpu codegen recursion shown in the backtrace and compare behavior with RUST_MIN_STACK set. Done means the direct-call version compiles without requiring the larger stack workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.