Azure / Azure/azure-sdk-for-rust

Eliminate workspace to make version updates easier, isolate service crate issues

Open
#4,791 3 comments 1 reaction 0 assignees View on GitHub
Azure.Core Client
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

## Summary

We are hitting duplicate core crate graphs when a crate needs to use in-repo `sdk/core` and `sdk/identity` sources while most of the workspace still depends on the latest shipped crate versions.

A concrete symptom is that commands like:

```bash
cargo tree -p azure_storage_blob_test -i azure_core
```

show multiple `azure_core` versions in the same package graph, which then causes type and trait mismatches (`TokenCredential`, `AsyncRawResponse`, `StatusCode`, error conversions, etc.).

## What we found

The split graph was introduced because different parts of the workspace resolved through different core stacks:

- shipped graph: `typespec 1.1.0`, `typespec_client_core 1.1.0`, `azure_core 1.1.0`, `azure_identity 1.0.0`
- local graph: in-repo `typespec 1.2.0-beta.1`, `typespec_client_core 1.2.0-beta.1`, `azure_core 1.2.0-beta.1`, `azure_identity 1.1.0-beta.1`

One observed case was:

- `azure_storage_blob_test` depended on workspace `azure_core 1.1.0`
- `azure_core_test` depended on local `azure_core 1.2.0-beta.1`
- `azure_core_test` also pulled local `azure_identity 1.1.0-beta.1`
- this introduced duplicate `typespec*` and `azure_core*` instances in downstream test graphs

## Experiments and results

### 1. Update only storage crates to use local `sdk/core` and `sdk/identity` paths

Tried changing `sdk/storage/*` manifests to point at local `sdk/core` / `sdk/identity` crates using path dependencies.

**Result:**

- This resolved the duplication for `azure_storage_blob_test` specifically.
- It did **not** solve the workspace-wide problem.
- `cargo build --all-features --all-targets` still failed in other crates (for example, `azure_security_keyvault_certificates`) because they were still mixing shipped and local core graphs.

Takeaway: fixing one area by hand is not enough; the problem is cross-workspace.

### 2. Keep shipped workspace dependency versions and add `[patch.crates-io]` for core crates

Tried a separate worktree experiment with `[patch.crates-io]` entries for:

- `typespec`
- `typespec_client_core`
- `typespec_macros`
- `azure_core`
- `azure_core_macros`
- `azure_identity`

First attempt added patch entries only.

**Result:**

- This did **not** work by itself.
- Cargo continued resolving many consumers to the published crates because the in-repo crate versions (`1.2.0-beta.1`, `1.1.0-beta.1`, etc.) did not satisfy the workspace's current shipped-version requirements.

Takeaway: patching alone does not help when the local crate versions do not match the requested versions.

### 3. Patch-based override with local crates advertising the shipped versions

In a second worktree experiment, we kept workspace dependency requirements at the currently shipped versions and added `[patch.crates-io]` overrides, but also changed the in-repo patched crates to advertise those same shipped versions.

Examples of the version alignment used in the experiment:

- `typespec` -> `1.1.0`
- `typespec_client_core` -> `1.1.0`
- `typespec_macros` -> `1.0.0`
- `azure_core` -> `1.1.0`
- `azure_core_macros` -> `1.0.0`
- `azure_identity` -> `1.0.0`

We also kept unpublished test-only crates as local workspace path dependencies.

**Result:**

- `azure_storage_blob_test` resolved to a single local `azure_core` graph.
- `azure_security_keyvault_certificates` resolved through the same local core stack instead of mixing local and published crates.
- `cargo build --all-features --all-targets` succeeded in the experiment.

Takeaway: this is the first approach that made the whole workspace build cleanly in the test setup.

## Important constraint

We do **not** want to ship `azure_core_test` as a supported published crate just to make this work.

The good news is that the successful patch-based experiment does **not** require publishing `azure_core_test`.

`azure_core_test` and `azure_core_test_macros` can remain unpublished local workspace crates. The important requirement is that the crates resolved from crates.io and overridden by `[patch.crates-io]` advertise versions that satisfy the workspace dependency requirements.

## Recommendation

The most promising direction looks like this:

1. Keep normal workspace dependency requirements pinned to the latest shipped core crate versions.
2. Keep `azure_core_test` and `azure_core_test_macros` as unpublished local workspace crates.
3. Add a coordinated `[patch.crates-io]` strategy for the published core stack (`typespec*`, `azure_core*`, `azure_identity`, and any other related crates that participate in the same public type graph).
4. Define a clear repo strategy for when local development should temporarily make the in-repo crates advertise the currently shipped versions so Cargo resolves the entire workspace to one local graph.

Open questions:

- Should this be a documented local-development workflow only?
- Should the version-alignment step be automated by a script/tool?
- Which crates must always participate together in the patched graph to avoid duplicate public types?

This issue is for tracking the resolution strategy and documenting a workspace-wide approach rather than continuing to patch individual service crates by hand.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the workspace manifests and the documented [patch.crates-io] experiment for the typespec, azure_core, and azure_identity crates. Use cargo tree -p azure_storage_blob_test -i azure_core and cargo build --all-features --all-targets to compare dependency graphs. Done means the workspace uses one local core graph while azure_core_test remains unpublished and local.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, developer-experience
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.