Add metrics support to Anthropic & Groq GenAI instrumentations

Problem

Observability is inconsistent across OpenTelemetry's GenAI instrumentations:

• OpenAI v2: Emits both spans AND metrics (operation duration, token usage histograms)
• Anthropic & Groq: Emit only spans—no metrics

This means teams using Anthropic Claude or Groq cannot track:

• Token consumption for cost analysis
• Operation latency distributions (p50, p99)
• Token quotas or rate limits

Currently, Anthropic and Groq instrumentations use TelemetryHandler which already contains InvocationMetricsRecorder, but token counts are never extracted and populated, so metrics are never recorded.

Evidence:
OpenAI v2 extracts result.usage.prompt_tokens and records metrics (link)
Anthropic does not extract tokens (link)
Groq instrumentation does not exist yet

Solution

1. Anthropic: Extract token counts from Message.usage response and populate invocation.input_tokens / invocation.output_tokens before handler.stop_llm()

   • Also capture Anthropic-specific cache tokens as metric attributes
   • Handle streaming via MessagesStreamWrapper token accumulation

2. Groq: Create new instrumentation package following Anthropic pattern

   • Extract from OpenAI-compatible response format
   • Record same metrics as Anthropic

Once tokens are populated on the invocation, TelemetryHandler.stop_llm() automatically calls InvocationMetricsRecorder.record() → metrics are emitted.

Result: Both instrumentations emit gen_ai.client.operation.duration and gen_ai.client.token.usage histograms with semantic attributes (model, provider, token_type).

---

Acceptance Criteria

• Anthropic instrumentation records gen_ai.client.operation.duration histogram
• Anthropic instrumentation records gen_ai.client.token.usage histogram (separate INPUT and COMPLETION records)
• Anthropic cache tokens (cache_creation_input_tokens, cache_read_input_tokens) stored as metric attributes
• Groq instrumentation created and records same metrics
• Token counts extracted correctly from both streaming and non-streaming responses
• Semantic attributes (model, provider, server address) present on metrics
• Unit tests ≥80% coverage for new code
• No breaking changes; existing spans unchanged
• Documentation examples show metrics usage

---

Implementation Plan

Phase 1: Anthropic Metrics (~200 loc)

1. Modify instrumentation-genai/opentelemetry-instrumentation-anthropic/src/opentelemetry/instrumentation/anthropic/messages_extractors.py

   • Add helper to extract token counts from Usage response
   • Expand UsageTokens dataclass for cache token fields

2. Modify patch.py

   • Populate invocation.input_tokens, invocation.output_tokens after response received
   • Store cache tokens in invocation.metric_attributes

3. Modify wrappers.py

   • Update MessagesStreamWrapper to accumulate token deltas from stream
   • Extract final counts at stream end

Phase 2: Groq Instrumentation (~350 loc, new package)

1. Create instrumentation-genai/opentelemetry-instrumentation-groq/
2. Create pyproject.toml (metadata, dependencies: groq, opentelemetry-api, opentelemetry-util-genai)
3. Create __init__.py with GroqInstrumentor class
4. Create patch.py wrapper for client.chat.completions.create()
5. Create utils.py with token extraction helper
6. Add basic unit tests

Phase 3: Testing (~200 loc)

• Unit tests for both using InMemoryMetricReader
• Assert histogram records and token counts
• Test streaming and non-streaming paths
• Integration tests (optional, with API keys)

---

Technical Details

Anthropic Response Format (already parsed):

usage = Message.usage
input_tokens = usage.input_tokens
output_tokens = usage.output_tokens
cache_creation_input_tokens = usage.cache_creation_input_tokens  # Anthropic-specific
cache_read_input_tokens = usage.cache_read_input_tokens  # Anthropic-specific

Groq Response Format (OpenAI-compatible):

usage = completion.usage
input_tokens = usage.prompt_tokens
output_tokens = usage.completion_tokens

Integration Point:

# In patch.py, before handler.stop_llm(invocation):
invocation.input_tokens = extracted_input_tokens
invocation.output_tokens = extracted_output_tokens
invocation.metric_attributes = {"gen_ai.usage.cache_creation.input_tokens": ...}

handler.stop_llm(invocation)  # Automatically records metrics

---

Why This Matters

• Cost tracking: Teams can attribute token spend to models/features
• Performance budgeting: Histogram percentiles reveal latency SLOs
• Multi-model comparison: Unified metrics across OpenAI, Anthropic, Groq
• Compliance: Required for some observability stacks (e.g., cost governance)

---

No Breaking Changes

• Existing Anthropic spans unchanged
• Metrics are additive (users without a meter provider see no impact)
• Zero performance overhead (token extraction on response path, reuses existing utilities)

---

Related

• Similar to #OpenAI v2 metrics implementation
• Uses existing InvocationMetricsRecorder from util-genai
• Aligns with OpenTelemetry GenAI Semantic Conventions

This work will be co-implemented by @JMGalvao.

Would you like to implement a fix?

None

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

Start with the Anthropic messages_extractors.py, patch.py, and wrappers.py files, then compare the OpenAI v2 metrics implementation. Run the planned unit tests with InMemoryMetricReader for streaming and non-streaming responses. Done means Anthropic and new Groq instrumentation emit duration and token histograms with the required attributes, tests, coverage, and documentation.