open-telemetry / open-telemetry/opentelemetry-python-genai

opentelemetry-util-genai: emit the per-operation execute_tool and invoke_agent metrics

Open
#576 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
39
Forks
63
Avg merge
1d 15h
Merged PRs (30d)
175

Description

What problem do you want to solve?

InvocationMetricsRecorder.record()
sends every invocation type to a single histogram, gen_ai.client.operation.duration.
Only WorkflowInvocation was given its own instrument; tool and agent invocations still
land in the client-operation metric, separated only by gen_ai.operation.name.
Four semconv instruments are missing:

Metric Bucket advisory
gen_ai.execute_tool.duration 0.01 … 81.92
gen_ai.invoke_agent.duration 0.1 … 409.6
gen_ai.invoke_agent.inference_calls 1 … 128
gen_ai.invoke_agent.tool_calls 1 … 128

All are Recommended, the highest level semconv defines for signals. Their attribute
sets also differ from the client metric's — gen_ai.execute_tool.duration requires
gen_ai.tool.name and carries no gen_ai.operation.name — so slicing one histogram by
operation name is not equivalent.
Two smaller conformance gaps in the existing workflow metric, if worth folding in:

  • it reuses the client bucket advisory (0.01 … 81.92) instead of its own (1 … 7200)
  • it sets gen_ai.operation.name, which is not in its attribute table
Describe the solution you'd like
  • Add the four instruments with their own bucket advisories.
  • Let each invocation declare its duration instrument and build metric attributes from
    that metric's table, instead of adding another record_<type>() per operation.
  • Scope inference_calls / tool_calls to the agent that issued them, so sub-agent
    calls are counted once.
Describe alternatives you've considered

No response

Additional Context

No response

Would you like to implement it?

Yes

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py at InvocationMetricsRecorder.record() and inspect how WorkflowInvocation declares and records its instrument. Add the four operation-specific instruments with the listed bucket advisories, use each metric's attribute table, and ensure agent inference_calls and tool_calls are scoped to their issuing agent. Done means tool and agent invocations emit the required metrics without relying on the client-operation histogram.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.