dotnet / dotnet/sdk

Enable shipping dotnet tools as workload packs (WorkloadPackKind.Tool)

Open
#53,615 4 comments 0 reactions 0 assignees View on GitHub
Area-Tools Area-Workloads
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

## Summary

Enable workload authors to include dotnet CLI tools in their workloads, so that `dotnet workload install maui` can also install associated CLI tools (e.g., a `maui` CLI tool). This leverages the existing `WorkloadPackKind.Tool` enum value that is already defined but not fully implemented.

## Motivation

Today, workloads can ship SDK packs, framework packs, library packs, and template packs — but **not CLI tools**. If a workload like MAUI wants to provide a command-line tool, users must install it separately via `dotnet tool install`. This creates a fragmented experience where:
- Users need to know about and separately install workload-associated tools
- Tool lifecycle is decoupled from the workload it belongs to
- Workload uninstall doesn't clean up associated tools
- Discoverability of workload tools is poor

## Current State (Groundwork Already Exists)

The SDK already has significant groundwork for this feature:

- ✅ `WorkloadPackKind.Tool` enum value exists (`src/Resolvers/Microsoft.NET.Sdk.WorkloadManifestReader/WorkloadPackKind.cs`) — since September 2020
- ✅ Path resolution maps to `tool-packs/{id}/{version}/` directories (`WorkloadResolver.cs:340-343`)
- ✅ Manifest parsing supports `"kind": "tool"` via `Enum.TryParse` (`WorkloadManifestReader.cs:526`)
- ✅ `GetInstalledWorkloadPacksOfKind(WorkloadPackKind.Tool)` API is ready
- ✅ `ShellShimRepository` and `ToolConfigurationDeserializer` infrastructure is available

---

## Phase 1 Implementation (Prototype Complete — Local Branch)

### Architecture

A working prototype has been built locally with the following components:

#### New Files (8)

| File | Purpose |
|------|---------|
| `src/Cli/dotnet/Commands/Workload/Install/WorkloadToolInstaller.cs` | Core shim management — reads `DotnetToolSettings.xml`, creates/removes shims via `ShellShimRepository`, handles repair/overwrite, user-global precedence |
| `src/Cli/dotnet/CommandFactory/CommandResolution/WorkloadToolsCommandResolver.cs` | CLI resolver with lazy in-memory cache — resolves `dotnet ` by scanning `tool-packs/` directories. O(n) first call, O(1) thereafter |
| `src/Cli/dotnet/Commands/Workload/WorkloadToolLocator.cs` | Discovery service — finds installed workload tools via `GetInstalledWorkloadPacksOfKind(Tool)`. Includes `WorkloadToolInfo` data class |
| `src/Cli/dotnet/Commands/Tool/List/ToolListWorkloadCommand.cs` | Handler for `dotnet tool list --workload` with table and JSON output |
| `test/dotnet.Tests/CommandTests/Workload/Install/GivenWorkloadToolInstaller.cs` | 10 unit tests for installer service |
| `test/dotnet.Tests/CommandTests/Tool/List/ToolListWorkloadCommandTests.cs` | 7 unit tests for list command |
| `test/TestAssets/TestProjects/SampleManifest/SampleWithToolPack.json` | Test manifest with tool pack |
| `documentation/general/workloads/workload-tool-packs.md` | Feature documentation |

#### Modified Files (9 source + auto-generated xlf)

| File | Change |
|------|--------|
| `FileBasedInstaller.cs` | `_workloadToolInstaller` field, post-install shim hook, rollback cleanup, GC shim removal |
| `DefaultCommandResolverPolicy.cs` | Added `WorkloadToolsCommandResolver` to resolver chain (after LocalTools, before Rooted) |
| `ToolListCommand.cs` | Routes `--workload`, validates conflicts with `--global`/`--local`/`--tool-path` |
| `ToolListJsonHelper.cs` | `WorkloadToolListJsonContract` + serializer context registration |
| `ToolListCommandDefinition.cs` | `WorkloadOption` definition |
| `CliCommandStrings.resx` | 7 new localized strings |
| `CommandDefinitionStrings.resx` | `ToolListWorkloadOptionDescription` |
| `GivenFileBasedWorkloadInstall.cs` | Integration test + `manifestPath` parameter support |
| `documentation/general/workloads/README.md` | Link to tool packs doc |

### Key Design Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Shim location | SDK-scoped `{dotnetRoot}/tools/` | Ties tool lifecycle to SDK installation |
| Precedence | User global tool wins | User intent > workload default; check `CliFolderPathCalculator.ToolsShimPath` before creating shim |
| Package format | Standard dotnet tool NuGet format | Reuses `DotnetToolSettings.xml`, no new format needed |
| Resolver position | After `LocalToolsCommandResolver`, before `RootedCommandResolver` | Local tools > workload tools > PATH |
| Resolver caching | Lazy in-memory `Dictionary` | Full dir scan + XML parse is O(p*v*t*r); cache makes subsequent calls O(1) |
| Filter matching | Exact `Equals` on PackId or CommandName | Consistent with `ToolListGlobalOrToolPathCommand` / `ToolListLocalCommand` patterns |
| Transactions | Per-pack with rollback | Shim creation inside per-pack transaction; rollback removes shims on failure |
| Installer scope | `FileBasedInstaller` only | macOS/Linux. MSI deferred to Phase 2. |

### Test Results

- **52 tests pass, 0 failures** (17 new + 35 existing, zero regressions)
- Build: 0 errors, 0 warnings

### Workload Manifest Example

```json
{
"workloads": {
"maui": {
"description": ".NET MAUI workload",
"packs": [
"Microsoft.Maui.Sdk",
"Microsoft.Maui.Templates",
"Microsoft.Maui.Tool"
]
}
},
"packs": {
"Microsoft.Maui.Tool": {
"kind": "tool",
"version": "9.0.0"
}
}
}
```

### CLI Usage

```bash
# Install workload (also installs tool shims)
dotnet workload install maui

# Use the tool
dotnet maui --help
maui --help # standalone shim also works

# List workload tools
dotnet tool list --workload
dotnet tool list --workload --format json
dotnet tool list --workload Microsoft.Maui.Tool

# Uninstall workload (cleans up tool shims)
dotnet workload uninstall maui
```

---

## Backward Compatibility

**TL;DR:** `WorkloadPackKind.Tool` exists in ALL workload SDKs (.NET 6+). Old SDKs silently extract tool packs to `tool-packs/` but create no shims — **no crashes, no errors**. On upgrade, `workload repair` creates shims for already-extracted packs.

| Step | Old SDK Behavior | Result |
|------|------------------|--------|
| Manifest parsing | `Enum.TryParse("tool")` succeeds | ✅ No error |
| Path resolution | Maps to `tool-packs/{id}/{version}/` | ✅ Correct |
| Download & extract | Normal NuGet download, directory extraction | ✅ Works |
| Shim creation | No code for this | ⚠️ No shim (silent) |
| Uninstall / GC | Pack deleted normally | ✅ Clean |

> ⚠️ Truly *unknown* kind values (e.g. `"kind": "plugin"`) **do** throw `WorkloadManifestFormatException`. But `Tool` is safe since it's been in the enum since Sept 2020.

## Porting to .NET 10

Only 2 files differ between `main` and `release/10.0.3xx`:
- `FileBasedInstaller.cs` — 3 lines (explicit `PackInfoJsonSerializerContext` vs reflection)
- `ToolListJsonHelper.cs` — source-gen serializer pattern

All other touched files are identical. Trivial cherry-pick with 1-2 merge conflict resolutions.

---

## Future Work (Phase 2+)

### 🔴 P0 — Required for Production

- [ ] **MSI installer support (Windows)** — `MsiInstallerBase` / `NetSdkMsiInstallerClient` needs tool shim creation as MSI post-install action. Consider whether shims should be created by the MSI custom action itself or by a post-MSI step in the .NET SDK installer.
- [ ] **End-to-end testing with real NuGet tool packages** — Current tests use mock downloaders. Need tests that download a real tool pack (e.g., `dotnet-ef`) from NuGet, install via workload manifest, and verify the tool is callable.
- [ ] **XLF localization review** — Run `/t:UpdateXlf` on modified `.resx` files to generate proper `needs-review-translation` states in `.xlf` files.
- [ ] **`workload update` tool handling** — Verify that `workload update` correctly handles tool pack version upgrades (remove old shim -> install new shim). Current implementation handles this via GC in `DeletePack`, but needs explicit testing.

### 🟡 P1 — Recommended Improvements

- [ ] **Resolver-based architecture (no shims)** — Both GPT 5.4 and Opus 4.5 independently proposed resolver-based alternatives that avoid shims entirely. Key benefits: simpler install (file copy only), MSI-friendly (no custom actions), cleaner conflict handling. Migration: add catalog + resolver behind feature flag -> dual support -> stop creating shims.
- [ ] **`dotnet tool run --workload`** — Allow explicit invocation of workload tools, similar to `dotnet tool run ` for local tools. Requires `ToolRunCommand` refactoring.
- [ ] **Conflict resolution policy** — When multiple workloads install tools with the same command name, current behavior is first-wins. Consider explicit policies (`FirstWins`, `LastWins`, `HigherVersionWins`, `Error`) configurable in the workload manifest.
- [ ] **`dotnet workload list` tool column** — Show which tools each workload provides in `dotnet workload list` output.
- [ ] **Tool pack validation in `workload repair`** — Verify tool shims are intact and re-create missing ones (currently supported but not explicitly tested).
- [ ] **Roll-forward support** — Tool packs should support roll-forward settings for the tool's target framework, similar to global tools.

### 🟢 P2 — Nice to Have

- [ ] **Per-pack `workloadtool.json` metadata** — Richer metadata format (explicit TFM/RID asset table, conflict policy, roll-forward settings) inside the tool pack itself. Avoids directory scanning for TFM/RID selection.
- [ ] **Derived `catalog.json`** — A computed catalog of all installed workload tools, rebuilt on pack mutations. Fast lookup for resolver and `dotnet tool list --workload`. Marks conflicting commands with `"state": "conflict"`.
- [ ] **`dotnet workload tool` subcommand** — Dedicated subcommand for managing workload tools (list, repair, diagnose).
- [ ] **Tab completion for workload tools** — Register workload tool commands with the shell completion system.
- [ ] **Telemetry** — Track workload tool usage (install/uninstall/invocation counts).
- [ ] **VS integration** — Ensure Visual Studio's workload installer also creates tool shims.

---

## Implementation Notes for Future Contributors

1. **`WorkloadToolInstaller.FindToolSettingsAndAssetDir()`** is the shared static helper that both installer and locator use. It returns both the settings XML path and the asset directory from the same TFM/RID folder, avoiding the mismatched-selection bug.

2. **Resolver chain order** in `DefaultCommandResolverPolicy.cs`: `Muxer -> DotnetToolsShim -> LocalTools -> **WorkloadTools** -> Rooted -> MuxerCli -> AppBase -> Path -> Current`. Moving WorkloadTools earlier/later changes precedence.

3. **Thread safety** — `WorkloadToolsCommandResolver` uses double-checked locking with `volatile` for the lazy cache. The cache is a `Dictionary` populated on first `Resolve()` call.

4. **Test infrastructure** — `MockNuGetPackageDownloader` creates placeholder files (no real `DotnetToolSettings.xml`). Tests that need tool settings create the XML manually. `DOTNET_CLI_HOME` env var redirects tool shim path for test isolation.

5. **`ShellShimRepository`** uses app-host on Windows and symlinks on Unix. On repair/update, existing shim must be removed before re-creating (it throws on duplicate).

6. **Alternative architectures** — See comment thread for detailed comparison of shim-based vs resolver-based approaches, proposed by independent multi-model review (GPT 5.4, Opus 4.5).

---

## Related Issues

- #52609 — Enables tool metapackages (complementary: bundling multiple NuGet tools). This issue is about shipping tools **inside workloads**.
- #23372 — Third-party workload support (broader scenario this feature enables)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.