[Migrated] External tracking: multiple targets in si... | Rust-GPU/rust-gpu

Nobody has claimed this yet.

Issue automatically imported from old repo: https://github.com/EmbarkStudios/rust-gpu/issues/791
Old labels: t: tracking issue,t: external
Originally creatd by khyperia on 2021-11-03T11:10:13Z

For a while, we've been talking about trying to take our absolutely horrifying, fragile, confusing hacks that let users embed rust-gpu programs in host programs and putting them into cargo itself. For example, this discussion (unfortunately in a private section of our discord).

The high-level goal is to basically remove spirv-builder and make its functionality part of cargo itself.

The extremely high level view of how it works, right now, is: "The user writes x86 crate, with a build.rs. There's a build-dependency on spirv-builder, which itself depends on rustc_codegen_spirv. Spirv-builder then invokes a nested cargo process (within the user's build.rs), passing in rustc_codegen_spirv as the backend to use, and using --target spirv-xyz. It then takes the resulting binary and spews it out of the user's build.rs to make it embeddable in the host x86 crate". We've spent countless, countless hours finding an extremely narrow, fragile path that makes it just barely work on most days, but when thursdays roll around, Thor sneezes and an undecipherable error from halfway around the world comes shooting out.

My theory of how this would be integrated into cargo (only a theory, could be implemented in some other way) is to make cargo aware of building multiple targets in a single cargo process invocation. That means that an x86 crate could declare a dependency on a SPIR-V crate (or a wasm crate, too!), and that dependency would get built, binary blob result gathered, and embedded into the host x86 crate. How those dependencies are specified and configured, how it's embedded, how everything works, are very very open questions.

This issue is for the high-level tracking of investigating what design work should be done, and how to go about implementing it - mainly so that this idea doesn't get lost in the depths of chat logs (again). I don't expect detailed design docs to be in this thread or whatever, I mostly intend this as a sticky note of "do this!!" ✨

Key Cargo issues tracking the development of new functionality to support this:

Start by reviewing the current spirv-builder and rustc_codegen_spirv flow described in the issue, then read the linked Cargo issues 9096 and 10030 and pull request 10061. Investigate how Cargo could build multiple target types in one process and document the dependency, configuration, embedding, and implementation design questions. Done means a concrete design direction and implementation plan for replacing the nested Cargo process.

Written by the indexing model from the issue text.

[Migrated] External tracking: multiple targets in single cargo process

Description

Contributor guide

First steps

Research direction

Assessment