Azure / Azure/azure-sdk-for-rust

Build native binaries with MS Rust

Closed
#5,237 1 comment 1 reaction 0 assignees View on GitHub
Client Cosmos EngSys
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

### Feature Summary

Build every publicly distributed Cosmos native-driver binary with the Microsoft Rust toolchain and record the exact toolchain in release evidence.

### Feature Description

Review guidance on #4991 established that the native binaries published to `Azure/azure-cosmos-driver` are not publicly releasable when built with the current upstream Rust setup. Update the production native-driver pipeline to use Microsoft Rust (`msrust`) rather than leaving this as a research task.

Repository prior art already establishes the core integration pattern on Daniel Jurek’s `onboard-msrust-toolchain` branch:

- `RustInstaller@1` installs MS Rust from the internal `ms-rust-tools` Azure Artifacts feed.
- An MS-specific `rust-toolchain.toml` pins an explicit `ms-prod-*` channel.
- `RUSTUP_EXE=msrustup` routes toolchain discovery through `msrustup`, because ordinary `rustup` cannot resolve Microsoft channels maintained outside its registry.
- The MS toolchain template omits `rust-std` from `components`; `msrustup` treats components as host components and reports `US.host_component_not_found` for that entry, while the host standard library already ships with the toolchain.

Implement that model for the Cosmos native-driver release path:

- Land or reuse a shared MS Rust installation path rather than duplicating installer and authentication logic in the Cosmos pipeline.
- Install MS Rust only in the internal/manual pipeline environment that can access the `ms-rust-tools` feed.
- Pin an approved `ms-prod-*` toolchain version and make its update policy explicit.
- Route all toolchain discovery and installation commands through the selected manager (`msrustup` for MS Rust, `rustup` otherwise).
- Configure required cross-compilation targets using the toolchain’s supported `targets` mechanism rather than declaring target standard libraries as host components.
- Build every release target in the supported Windows, Linux glibc, Linux musl, and macOS matrix with MS Rust. If a target is unavailable, fail closed and document the release gap rather than silently falling back to upstream Rust.
- Verify compatibility with the target-specific C compilers, linkers, `cargo-auditable`, bindgen, native static-library discovery, SBOM generation, and the downstream Go link smoke tests.
- Record the MS Rust channel, `rustc -Vv`, Cargo version, toolchain manager, target, linker, and source commit in per-target metadata and consolidated provenance.
- Add Pester coverage that detects an upstream-Rust fallback, an unpinned channel, missing target support, or provenance that omits the MS Rust identity.
- Run the complete native-driver pipeline manually and verify that its generated downstream PR contains only MS Rust-built artifacts.

### Use Case

The Rust pipeline publishes prebuilt native libraries that become transitive runtime dependencies of Cosmos Go V2 applications. Microsoft must be able to release, service, and attest those exact bytes under its approved Rust toolchain policy without requiring customers to install Rust.

### Alternatives

Continue building with upstream Rust and block public publication, or obtain a documented release-policy exception. The pipeline must not mix MS Rust and upstream Rust artifacts within one release.

### Additional Context

Follow-up to #4991 and related to #4967, #4968, #4969, and #4990.

Relevant repository prior art:

- `origin/djurek/onboard-msrust-toolchain`
- `bc4f31cf2` — first internal MS Rust toolchain integration
- `c7c9ae6db` — valid runtime matrix expression for the MS Rust leg
- `767ca90d3` — remove incompatible `rust-std` host-component declaration
- `dc2cb68f5` — route toolchain queries through `RUSTUP_EXE`
- `eng/pipelines/templates/steps/use-rust.yml` — Cargo feed configuration and authentication used by #4991; this is complementary to, not a replacement for, installing MS Rust

### Definition of Done

- The production native-driver pipeline installs and uses a pinned, approved `ms-prod-*` toolchain for every artifact intended for public distribution.
- No release artifact can be produced by silently falling back to upstream Rust.
- Every supported target builds, generates metadata and SBOM/provenance evidence, and passes its downstream Go link or load validation with MS Rust.
- The published provenance identifies the Microsoft Rust toolchain and all target/linker inputs needed to reproduce the build.
- Automated tests fail when MS Rust is absent, the channel is unpinned, target support is missing, or release evidence does not identify MS Rust.
- A manual end-to-end pipeline run successfully opens a downstream `Azure/azure-cosmos-driver` PR containing only verified MS Rust-built binaries.
- Any target unsupported by MS Rust is removed from the releasable matrix or blocked with an owner-approved, documented disposition.

Contributor guide

Open the contributing guide

Research direction

Start with eng/pipelines/templates/steps/use-rust.yml and the onboard-msrust-toolchain branch, including commits bc4f31cf2, c7c9ae6db, 767ca90d3, and dc2cb68f5. Trace the production native-driver pipeline and its Windows, Linux glibc, Linux musl, and macOS matrix, then run its Pester and downstream Go link validations. Done means every releasable target uses pinned MS Rust and emits complete toolchain, target, linker, SBOM, and provenance evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go, rust
Domain
build-system, ci-cd, devops, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.