Azure / Azure/azure-sdk-for-rust
Eliminate workspace to make version updates easier, isolate service crate issues
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 109
Description
## Summary
We are hitting duplicate core crate graphs when a crate needs to use in-repo `sdk/core` and `sdk/identity` sources while most of the workspace still depends on the latest shipped crate versions.
A concrete symptom is that commands like:
```bash
cargo tree -p azure_storage_blob_test -i azure_core
```
show multiple `azure_core` versions in the same package graph, which then causes type and trait mismatches (`TokenCredential`, `AsyncRawResponse`, `StatusCode`, error conversions, etc.).
## What we found
The split graph was introduced because different parts of the workspace resolved through different core stacks:
- shipped graph: `typespec 1.1.0`, `typespec_client_core 1.1.0`, `azure_core 1.1.0`, `azure_identity 1.0.0`
- local graph: in-repo `typespec 1.2.0-beta.1`, `typespec_client_core 1.2.0-beta.1`, `azure_core 1.2.0-beta.1`, `azure_identity 1.1.0-beta.1`
One observed case was:
- `azure_storage_blob_test` depended on workspace `azure_core 1.1.0`
- `azure_core_test` depended on local `azure_core 1.2.0-beta.1`
- `azure_core_test` also pulled local `azure_identity 1.1.0-beta.1`
- this introduced duplicate `typespec*` and `azure_core*` instances in downstream test graphs
## Experiments and results
### 1. Update only storage crates to use local `sdk/core` and `sdk/identity` paths
Tried changing `sdk/storage/*` manifests to point at local `sdk/core` / `sdk/identity` crates using path dependencies.
**Result:**
- This resolved the duplication for `azure_storage_blob_test` specifically.
- It did **not** solve the workspace-wide problem.
- `cargo build --all-features --all-targets` still failed in other crates (for example, `azure_security_keyvault_certificates`) because they were still mixing shipped and local core graphs.
Takeaway: fixing one area by hand is not enough; the problem is cross-workspace.
### 2. Keep shipped workspace dependency versions and add `[patch.crates-io]` for core crates
Tried a separate worktree experiment with `[patch.crates-io]` entries for:
- `typespec`
- `typespec_client_core`
- `typespec_macros`
- `azure_core`
- `azure_core_macros`
- `azure_identity`
First attempt added patch entries only.
**Result:**
- This did **not** work by itself.
- Cargo continued resolving many consumers to the published crates because the in-repo crate versions (`1.2.0-beta.1`, `1.1.0-beta.1`, etc.) did not satisfy the workspace's current shipped-version requirements.
Takeaway: patching alone does not help when the local crate versions do not match the requested versions.
### 3. Patch-based override with local crates advertising the shipped versions
In a second worktree experiment, we kept workspace dependency requirements at the currently shipped versions and added `[patch.crates-io]` overrides, but also changed the in-repo patched crates to advertise those same shipped versions.
Examples of the version alignment used in the experiment:
- `typespec` -> `1.1.0`
- `typespec_client_core` -> `1.1.0`
- `typespec_macros` -> `1.0.0`
- `azure_core` -> `1.1.0`
- `azure_core_macros` -> `1.0.0`
- `azure_identity` -> `1.0.0`
We also kept unpublished test-only crates as local workspace path dependencies.
**Result:**
- `azure_storage_blob_test` resolved to a single local `azure_core` graph.
- `azure_security_keyvault_certificates` resolved through the same local core stack instead of mixing local and published crates.
- `cargo build --all-features --all-targets` succeeded in the experiment.
Takeaway: this is the first approach that made the whole workspace build cleanly in the test setup.
## Important constraint
We do **not** want to ship `azure_core_test` as a supported published crate just to make this work.
The good news is that the successful patch-based experiment does **not** require publishing `azure_core_test`.
`azure_core_test` and `azure_core_test_macros` can remain unpublished local workspace crates. The important requirement is that the crates resolved from crates.io and overridden by `[patch.crates-io]` advertise versions that satisfy the workspace dependency requirements.
## Recommendation
The most promising direction looks like this:
1. Keep normal workspace dependency requirements pinned to the latest shipped core crate versions.
2. Keep `azure_core_test` and `azure_core_test_macros` as unpublished local workspace crates.
3. Add a coordinated `[patch.crates-io]` strategy for the published core stack (`typespec*`, `azure_core*`, `azure_identity`, and any other related crates that participate in the same public type graph).
4. Define a clear repo strategy for when local development should temporarily make the in-repo crates advertise the currently shipped versions so Cargo resolves the entire workspace to one local graph.
Open questions:
- Should this be a documented local-development workflow only?
- Should the version-alignment step be automated by a script/tool?
- Which crates must always participate together in the patched graph to avoid duplicate public types?
This issue is for tracking the resolution strategy and documenting a workspace-wide approach rather than continuing to patch individual service crates by hand.
Contributor guide
Research direction
Start by reviewing the workspace manifests and the documented [patch.crates-io] experiment for the typespec, azure_core, and azure_identity crates. Use cargo tree -p azure_storage_blob_test -i azure_core and cargo build --all-features --all-targets to compare dependency graphs. Done means the workspace uses one local core graph while azure_core_test remains unpublished and local.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system, developer-experience
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100