Improving dotnet CLI token efficiency

I've worked on a number of prototypes that intended to improve CLI token efficiency, like [`dotnet-inspect`](https://github.com/richlander/dotnet-inspect) and [markdown-table-logger](https://github.com/richlander/markdown-table-logger). They have been well-intended but pretty clearly "solution hypothesis". I'm now working the problem hypothesis side. It brought me to the model service pricing tables and the distinction between tokens. My new theory is that the biggest leverage the dotnet CLI has on agentic cost is to reduce the generation of thinking tokens. A new project should aim to reduce thinking tokens per accomplished task.

Put another way, the projects I did before are _probably_ good, but were missing a success metric. That's why they haven't moved past the prototype stage. A good success metric would motivate changes in the CLI. I have a separate set of ideas on the actual integration; I'll leave those for another issue. This proposal motivates the integration question without defining the gate — the harness design itself is a follow-up.

## Pricing

Frontier model pricing is largely the same across vendors: cached input is cheap, fresh input is more expensive, and output is dramatically more expensive than either. The Anthropic pricing table below is illustrative; OpenAI and Google publish comparable input/output spreads. The structural argument here doesn't depend on the specific vendor or model.

| Model | Base Input Tokens | 5m Cache Writes | 1h Cache Writes | Cache Hits & Refreshes | Output Tokens |
| ------------------------------------------------------------------------ | ----------------- | --------------- | --------------- | ---------------------- | ------------- |
| Claude Opus 4.7 | $5 / MTok | $6.25 / MTok | $10 / MTok | $0.50 / MTok | $25 / MTok |
| Claude Opus 4.6 | $5 / MTok | $6.25 / MTok | $10 / MTok | $0.50 / MTok | $25 / MTok |
| Claude Sonnet 4.6 | $3 / MTok | $3.75 / MTok | $6 / MTok | $0.30 / MTok | $15 / MTok |
| Claude Sonnet 4.5 | $3 / MTok | $3.75 / MTok | $6 / MTok | $0.30 / MTok | $15 / MTok |
| Claude Haiku 4.5 | $1 / MTok | $1.25 / MTok | $2 / MTok | $0.10 / MTok | $5 / MTok |

Source: [Anthropic API pricing](https://platform.claude.com/docs/en/about-claude/pricing).

This is telling us pricing varies by operation and that we should avoid hitting the output-rate bucket as much as possible. For Opus 4.7, there is a 50x difference between the cheapest and most expensive bucket. That means 50k tokens from the input cache cost the same as 1k output tokens.

However, dotnet CLI output only ever contributes to input tokens and that quickly becomes very cheap. So, we're covered! Maybe not.

## Thinking

Thinking tokens are a special case. They are tokens the model generates as internal reasoning before the visible response. Critically, they are charged at the output rate. In some cases, depending on the client, they are not preserved. That's mostly important for correctly measuring session costs. The pathological case is that there is a complexity that gets worked out in every turn with thinking and is never preserved.

The model reasoning levels (Low, Medium, High, ...) are caps on the number of tokens to spend reasoning or thinking. That explains why the table doesn't have separate reasoning level costs.

**Hypothesis 1:** Thinking tokens are the session cost driver, particularly at `high` and `xhigh` effort.

**Hypothesis 2:** Disabling thinking on noise-filtering-dominated tasks reduces cost without proportional quality loss.

### Skills as cost shifting

Thinking pricing puts the [invention of skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) in a different light. They were sold as a kind of DX innovation and domain-knowledge extension. That's all true. They also enable a kind of pricing arbitrage: they slide costly thinking tokens to the input side. It also explains why skills are not framed as casually-written sticky-notes but as engineering artifacts. [`dotnet/skills`](https://github.com/dotnet/skills) is our own skills repository.

Imagine you successfully add 100k skill input tokens in exchange for removing 100k output tokens. Let's assume the 100k tokens are read 10 times (1x base input + 9x cache). That's $0.95 in increased input costs to save $2.50 on the output side. Good. If you remove 500k output tokens, you save $12.50 on the output side. Great! If you do that every session, the savings accumulate. Skills often enable tasks the model wouldn't have attempted rather than displacing output 1:1, so the unit economics are not quite so convenient in practice.

The lesson is that skills are pricing arbitrage — they move spend from the expensive output bucket to the cheaper input bucket. That's what makes them worth the engineering investment.
## Irregular logs

The driving intuition of this work is that dotnet CLI output is verbose and irregular. It's difficult to define a regular format for any one command, and command output formats differ greatly. I've often used [jellyfin/jellyfin](https://github.com/jellyfin/jellyfin) as a test bed. Here is a representative excerpt from `dotnet build Jellyfin.sln` at [`e2fe5c1`](https://github.com/jellyfin/jellyfin/commit/e2fe5c163ae7ca12e862df70fb8d6caa14e86ef2) (2026-05-21) — 215 warnings, 0 errors, ~540 lines of output, ~22 seconds wall time.

Prompt: "Fix all the warnings in this repo. Do not fix or suppress warnings where the fix would affect API compatibility. Report before and after warning counts by warning types. Do the work in a worktree, with the branch including the LLM model name and reasoning level."

```text
$ dotnet build Jellyfin.sln
Determining projects to restore...
Restored /home/rich/git/jellyfin/src/Jellyfin.CodeAnalysis/Jellyfin.CodeAnalysis.csproj (in 395 ms).
Restored /home/rich/git/jellyfin/src/Jellyfin.Extensions/Jellyfin.Extensions.csproj (in 395 ms).
... 38 more "Restored" lines ...
/home/rich/git/jellyfin/MediaBrowser.Model/Configuration/TypeOptions.cs(322,25): warning CA1819: Properties should not return arrays (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca1819) [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
/home/rich/git/jellyfin/MediaBrowser.Model/Configuration/TypeOptions.cs(324,25): warning CA1819: Properties should not return arrays (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca1819) [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
/home/rich/git/jellyfin/MediaBrowser.Model/Configuration/TypeOptions.cs(326,30): warning CA1819: Properties should not return arrays (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca1819) [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
... 50+ more CA1819 warnings from MediaBrowser.Model ...
MediaBrowser.Controller -> /home/rich/git/jellyfin/MediaBrowser.Controller/bin/Debug/net10.0/MediaBrowser.Controller.dll
Jellyfin.MediaEncoding.Hls -> /home/rich/git/jellyfin/src/Jellyfin.MediaEncoding.Hls/bin/Debug/net10.0/Jellyfin.MediaEncoding.Hls.dll
/home/rich/git/jellyfin/Emby.Server.Implementations/Data/SqliteExtensions.cs(75,13): warning CA2100: Review if the query string passed to 'string SqliteCommand.CommandText' in 'Execute', accepts any user input (...) [/home/rich/git/jellyfin/Emby.Server.Implementations/Emby.Server.Implementations.csproj]
Emby.Server.Implementations -> /home/rich/git/jellyfin/Emby.Server.Implementations/bin/Debug/net10.0/Emby.Server.Implementations.dll
Jellyfin.Server -> /home/rich/git/jellyfin/Jellyfin.Server/bin/Debug/net10.0/jellyfin.dll

Build succeeded.

/home/rich/git/jellyfin/MediaBrowser.Model/Channels/ChannelQuery.cs(15,30): warning CA1819: Properties should not return arrays (...) [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
/home/rich/git/jellyfin/MediaBrowser.Model/Entities/IHasProviderIds.cs(14,36): warning CA2227: Change 'ProviderIds' to be read-only by removing the property setter (...) [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
... 100+ more warnings ...

215 Warning(s)
0 Error(s)

Time Elapsed 00:00:21.54
```

Here's a single line from the log:

```text
/home/rich/git/jellyfin/MediaBrowser.Controller/Resolvers/IItemResolver.cs(50,41): warning CA1002: Change 'List' in 'MultiItemResolverResult.ExtraFiles' to use 'Collection', 'ReadOnlyCollection' or 'KeyedCollection' (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca1002) [/home/rich/git/jellyfin/MediaBrowser.Controller/MediaBrowser.Controller.csproj]
```

Several distinct challenges are visible here. They probably have different fixes and cost impacts:

- **Per-line redundancy.** Every diagnostic carries an absolute file path, then the same project's `.csproj` absolute path in brackets, plus a documentation URL. The owning project is implied two times per line.
- **No clear column demarcator.** If anything, `:` is the demarcator, but it is also overused.
- **Repetition without aggregation.** `CA1819` fires dozens of times against `MediaBrowser.Model`. Nothing in the output tells you "this is the same rule firing N times" — the reader has to do that grouping.
- **Interleaved per-project output.** Diagnostic lines from one project are mixed with `Project -> .../X.dll` success lines from other projects. There is no visual grouping by project.
- **Non-deterministic ordering.** `Build succeeded.` is followed by another ~200 lines of warnings, including warnings from projects whose warnings already appeared earlier. The boundary line does not mean what it appears to mean. A reader who stops at the first apparent summary will miss most of the output.
- **Bi-modal logs.** Terminal logger was built for humans and the old log (`--tl:off`) is generally bad for all users. Neither was designed for agents. Ironically, some agents use the new log and others use the old one. The old one is being demonstrated above (generated by an agent with no instruction on which logger to use).
- **Severity buried mid-line.** `warning CAXXXX:` is in the middle of each line, not at the start. A line scanner has to find it before it knows whether the line matters.
- **Warning debt.** This project has targeted [warning reduction](https://github.com/jellyfin/jellyfin/issues/2149) as a multi-year project. That means that most of the warnings are a waste of input and reasoning tokens for agentic work.

LLMs cope with this output, but each defect likely adds a reasoning task before any signal can be extracted. That cost shows up in thinking tokens, not in the visible output.

**Hypothesis 3:** Log cleanliness moves cost between three columns, but only one of them shows up in standard session metrics:

- **Visible output (the answer):** approximately stable. The agent reaches the same conclusion regardless of log cleanliness.
- **Turn count:** may rise with noisy logs at low thinking, as the agent compensates for the missing reasoning with iteration. _Visible_ in session metrics.
- **Reasoning tokens:** may rise with noisy logs at high thinking. _Invisible_ in standard metrics; very much _visible in agent cost_ — reasoning is billed at the output rate. Only surfaced separately via per-call telemetry.

The cost of dirty logs is paid in either currency — turns at low thinking, reasoning at high thinking. Any metric that only counts output tokens will miss the cleanup opportunity at both ends.

As a case in point, in one of my test run, I saw Copilot need to capture logs to a file and then use Unix core tools to slice and dice it into manageable bits. It's clearly capable. The cost is in time (more turns) and tokens.

Example:

```
● Build to see warnings (shell)
│
│ Output too large to read at once (23.6 KB). Saved to:
│ /tmp/copilot-tool-output-1779473829140-5t60bj.txt
│ Consider using tools like grep (for searching), head/tail (for viewing
│ start/end), view with view_range (for specific sections), or jq (for
│ JSON) to examine portions of the output.
│
│ Preview (first 500 chars):
│ /home/rich/git/jellyfin/MediaBrowser.Model/Drawing/ImageOrientation.cs(5,
│ 17): warning CA1008: Add a member to ImageOrientation that has a value of
│ zero with a suggested name of 'None' (https://learn.microsoft.com/dotnet
│ /fundamentals/code-analysis/quality-rules/ca1008)
│ [/home/rich/git/jellyfin/MediaBrowser.Model/MediaBrowser.Model.csproj]
│ /home/rich/git/jellyfin/MediaBrowser.Model/Channels/ChannelQuery.cs(15,30
│ ): warning CA1819: Properties should not return arrays
│ (https://learn.microsoft.com/dot
│
│
│ cd /home/rich/git/jellyfin && dotnet build 2>&1 | grep -E "warning|error" |
│ head -80
└ 7 lines

● Summarize warning types (shell)
│
│ 65 warning CA1819
│ 5 warning CA1008
│ 4 warning CA2227
│ 3 warning CA1721
│ 2 warning CA1027
│ 1 warning CA1069
│
│
│ cat /tmp/copilot-tool-output-1779473829140-5t60bj.txt | grep "warning" |
│ grep -oP "warning \w+" | sort | uniq -c | sort -rn
└ 7 lines
```

**Hypothesis 4:** Three characteristics specific to the .NET CLI — **verbose per-line output, irregular structure, and lack of a query interface** — are the addressable causes of noise-induced thinking in .NET-centric agent sessions. These are the defects an agent-mode (and tools like `dotnet-inspect`) directly address.

It's possibly even worse than it appears. In the normal case, the model needs to reason about the last `dotnet build` result. Imagine a prompt that causes the model to compare across several recent builds. _Accessing_ the input cache is effectively free while _reasoning_ about it is far from that.

### Note: visible-token share misses thinking

When I previously measured dotnet token share of total session tokens, the result was small (6–10% of fresh input). I took that as evidence *against* the hypothesis. On further analysis, a small share is plausibly evidence *for* it: if irregular logs drive thinking, the dirtier the logs, the more thinking they produce, and the smaller dotnet's *visible* share will look in the total. Any measurement that doesn't separate reasoning from visible output will under-count the cleanup opportunity. Anyone arguing that "dotnet share is small therefore the proposal targets a minor lever" is looking at the wrong column. At least, that's the thinking to prove.

## Cost efficiency

If that's all true, then `xhigh` thinking has high cost because you are hitting the output-rate bucket a lot. The model is constantly needing to reason about irregular logs to find signal among the noise. Imagine that we improve CLI output and can reduce the median turn thinking cost by 50% while maintaining overall solution performance. That's super good and interesting, but not the killer point.

The more we can reduce median thinking cost, the more that organizations will be happy to pay for developers to use higher thinking tiers. The intent is that thinking cost transitions from an overall step function to a turn-specific performance reasoning boost that delivers actual value. The assumption is that high-value peaks are likely not evident today because there are no valleys.

The aim isn't to reduce thinking overall — it's to reduce thinking *on the noise* so the budget can be spent on the actual goal. Reasoning that correlates with task difficulty is valuable; reasoning that correlates with log dirtiness is overhead. H5 below depends on this distinction: it's only an unlock if higher tiers can be made cheap on noise, not cheaper overall.

**Hypothesis 5:** Reducing median turn thinking cost enables using higher thinking tiers without exponential cost increase.

The median value might not tell the whole story. Thinking is more quota than allotment. The capacity-hit rate is worth measuring. It's a signal that logs may be blowing the budget.

**Hypothesis 6:** Cleanup reduces the rate of thinking cap-hit turns.

## Early findings

The hypotheses above are informed in part by a small set of jellyfin warning-cleanup runs.

The strongest single exhibit is the Opus 4.7 thinking tier ladder. Three Opus runs on warning-cleanup prompts (the `off` and `xhigh` runs share the same prompt and starting state; `high` is from an earlier session with a slightly different phrasing):

| Opus 4.7 tier | Turns | Per-turn output | Total reasoning | Total cost |
| ------------- | ----: | --------------: | -------------------: | ---------: |
| off | 31 | 347 | 0 | $1.20 |
| high | 22 | 610 | ~2,000 (chars-est) | $1.26 |
| xhigh | 56 | 607 | ~7,800 (auth + est) | $4.69 |

The off→high transition raised both per-turn output and reasoning. **The high→xhigh jump kept per-turn visible output flat at ~610 tokens — the growth was all in reasoning**: total reasoning rose from ~2,000 to ~7,800 tokens (~4×), turn count nearly tripled, and total cost rose proportionally. The extra thinking budget at xhigh went somewhere; it didn't go into longer or better answers.

The `off` and `xhigh` runs above, plus a GPT 5.5 xhigh control, ran the same prompt (shared above) against the same starting state, each agent working in its own git worktree so the committed diffs are directly comparable:

| Run | Model + tier | Turns | Per-turn output | Total reasoning | Wall | Total cost |
| --- | --------------------------------------------------- | ----: | --------------: | -----------------------------: | ------: | ---------: |
| A | Opus 4.7 thinking **off** (`reasoning_effort=none`) | 31 | 347 | 0 | 4m 42s | **$1.20** |
| B | Opus 4.7 **xhigh** | 56 | 607 | ~7,800 (auth + chars-estimate) | 14m 18s | **$4.69** |
| C | GPT 5.5 **xhigh** | 30 | 603 | 5,680 (authoritative) | 8m 39s | **$2.04** |

_Cost estimates sum all four billing buckets per session — uncached input, cache-read, cache-write, and output (including reasoning) — at current public per-MTok rates: Opus 4.7 $5 / $0.50 / $6.25 / $25; GPT 5.5 $5 / $0.50 / — / $30. Opus reasoning is partly estimated from disk-captured reasoning text because Anthropic bundles thinking into the output-token bill; GPT's reasoning is reported separately and is authoritative. GPT 5.5 also dispatched 8 Opus sub-agent calls during its run, included in the session total; the per-turn output figure (603) is over the 30 main-thread turns only._

**All three produced identical task outcomes.** Same five fixes to the same two files: three `CA2100` suppressions on `SqliteExtensions.Query/Execute/PrepareStatement` (helpers that intentionally execute caller-supplied SQL by design), plus two `CA2007` `ConfigureAwait(false)` additions in `InstallationManager`. All correctly identified the remaining ~210 CA warnings as API-breaking and left them in place. The committed diffs differ only in code-formatting indentation and prose wording of `Justification` strings. There was no meaningful quality difference between $1.20 and $4.69 — Opus xhigh paid ~$3.49 more (and ran 3× the wall clock, 1.8× the turns) for the same code changes.

Wall-clock time scales with cost — ~3× longer for Opus xhigh than Opus off — and time-to-first-token roughly doubles once thinking turns on (~4s → ~8s). Cleaner CLI output should reduce both cost and per-call latency simultaneously; the latency reduction is the more visible UX improvement.

### What this data does and doesn't show

**Strong evidence**: reasoning was spent without changing the answer on this task. Same prompt, same starting state, same committed diff, 4× cost spread.

**Suggestive evidence**: the noise in the dotnet output is involved. The model's captured reasoning surface explicitly contains build-output content (the agent reasons about warning counts, CA codes, and how to filter the build output); total reasoning grows with tier on identical input; visible output per turn stops growing once thinking is on.

**What we don't have**: a controlled measurement that varies log cleanliness at fixed model and tier. That isolation is what the testing harness exists to provide. Until then, the cleanup-causes-the-reasoning-reduction link is a plausible mechanism the data is consistent with — not a measured causal chain. A reader who accepts only one claim from this section should accept "thinking was spent without improving the answer," not "logs are the reason."

## Opportunity

The pitch at the start was to add x input tokens in trade for y fewer thinking tokens. That's the skill proposition. This proposal is quite different, with the idea that x _fewer_ input tokens will lead to y fewer thinking tokens.

We don't solely need to tighten up the logs. We need redesign the interaction model. The premise of one log per build is the most problematic aspect. You can see that in the Jellyfin logs. The agent is attempting to derive a higher level view. The final vision is logs that are queryable with both pre-packaged and configurable views.

The dotnet-inspect already does this. You can watch the agent quickly adapt to expert use of the tool. The tool was built as a query engine over domain data with an eye to applying the model to the dotnet CLI. It works. We just need to refine the reward metric.

If H3 and H6 don't replicate at scale, the redesign argument weakens and the next move is verbosity-only cleanup on the existing CLI rather than a queryable replacement.
## Project

The project is to create a testing harness that can validate these hypotheses, using a combination of `dotnet-inspect` and the actual dotnet CLI. It may be easier to prove out the base idea with dotnet-inspect. It already has the end-state query system. We can easily add a verbose irregular mode to the tool as a proxy for the dotnet CLI.

The primary measures will be:

- Median turn thinking token cost
- Thinking capacity hit turns
- Total thinking token cost
- Total output token cost
- Task success

Of these, **task success** is the variable that makes the others interpretable. A defensible per-task-class rubric is the harness's most important design decision — without it the optimization degenerates ("reduce thinking by disabling thinking"). The rubric will be defined as part of the harness, not in this proposal.

The outcome we're looking for is being able to significantly reduce thinking tokens generally and median thinking tokens in particular. The two should correlate.

Assuming we see positive results, the next step will be changing the dotnet CLI in alignment with the findings.

I'll file follow-up issues as this work progresses — for the harness design itself, for measurement results against the hypotheses above, and for any concrete cleanup proposals that the data justifies. This issue is the starting point, not the full plan.

The issue names no SDK file, test, or implementation entry point. Start by reviewing the linked dotnet-inspect, markdown-table-logger, and dotnet/skills projects, then determine how a measurable reduction in thinking tokens could be evaluated; the current issue does not define a completion gate or harness.

Written by the indexing model from the issue text.