dotnet / dotnet/extensions

Add strongly-typed UsageDetails for cache write tokens

Open
#7,625 1 comment 2 reactions 0 assignees View on GitHub
untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

Anthropic and OpenAI both now expose details on tokens that are written to the prompt cache. However, currently M.E.AI only exposes a strongly-typed property for cache reads, not cache writes. We should expose it. Additionally, otel defines in the genai semantic convention how cache writes should be emitted, just as it does for cache reads... we should make sure the data flows correctly.

Contributor guide

Open the contributing guide

Research direction

Start by locating M.E.AI's existing strongly typed cache-read UsageDetails property and the code that maps usage data to OpenTelemetry GenAI semantic conventions. Trace the cache-write data path and compare it with the cache-read path. Done means cache-write details are strongly typed and emitted correctly according to the stated convention; add or update the relevant tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.