traceloop / traceloop/openllmetry
π Feature: CrewAI: instrument BaseTool.run to emit tool-execution spans
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.4k
- Forks
- 1.1k
- Avg merge
- 8d 14h
- Merged PRs (30d)
- 2
Description
Which component is this feature for?
All Packages
π Feature description
Component: CrewAI Instrumentation β not in the dropdown above, so I selected "All Packages."
opentelemetry-instrumentation-crewai==0.62.3 creates no span for tool execution. A trace consumer can see which tools an agent had, but not which ran, in what order, with what arguments, or what they returned.
CrewAIInstrumentor._instrument wraps four call sites, and _uninstrument mirrors them β instrumentation.py#L87-L94: Crew.kickoff, Agent.execute_task, Task.execute_sync, LLM.call. crewai.tools.base_tool BaseTool.run is not among them.
Tools reach the trace only through _serialize_tools, which filters each tool to name and description β crewai_span_attributes.py#L161-L177 β landing as crewai.agent.tools and crewai.task.tools.
π€ Why is this feature needed ?
Tool-level visibility is the part of an agent trace that shows what the agent actually did. Without a span per tool execution there's no tool latency, no tool error surface (a raising tool appears only as the agent span's status), and no record of the arguments a tool was called with.
Verified live β crewai==1.15.18, traceloop-sdk==0.62.3, opentelemetry-instrumentation-crewai==0.62.3, Python 3.12.3. A two-tool crew (fetch_ticket, send_email) where both tools demonstrably executed; the full span inventory for the run:
3 openai.chat
1 Support Responder.agent
1 <task description>.task
1 crewai.workflow
That's a name-agnostic inventory of every span emitted, so it holds regardless of what a tool span would have been called. Nothing corresponds to a tool invocation.
The existing tool attributes don't compensate, because they're snapshotted at span start via CrewAISpanAttributes(span, instance) before the tools run. After both tools completed successfully, the exported spans still read:
crewai.agent.tools_results = []
crewai.task.used_tools = 0
crewai.task.tools_errors = 0
So the counters that exist are also wrong β arguably a separate defect, but I mention it here because it's the obvious answer to "isn't the tool data already on the agent span?"
Your own RFC #3460 already specifies this span:
Description: Execution of a tool, function, or capability by an agent.
Span Kind:
CLIENTRequired Attributes:
Attribute Type Description Examples gen_ai.tool.namestring Tool/function name "web_search","calculator","read_file"gen_ai.tool.typestring Tool category "api","function","code","mcp","native","browser"gen_ai.operation.namestring Operation performed "execute","invoke","call"
Its framework examples read "All frameworks: Function calling, custom tools, built-in tools", then name Agno, Haystack and Smolagents. CrewAI isn't named, which is why this is a feature request against your documented direction rather than a spec-violation bug.
βοΈ How do you aim to achieve this?
Wrap crewai.tools.base_tool BaseTool.run alongside the existing wraps, emitting one span per tool invocation with gen_ai.tool.name, gen_ai.tool.type and gen_ai.operation.name, plus arguments and result where content capture is enabled.
I'd default to the RFC #3460 shape (gen_ai.tool.execute, CLIENT) since it's your documented direction, unless you'd rather track the upstream OTel GenAI execute_tool span β specified at https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md (note the GenAI conventions moved out of the main semantic-conventions repo). Happy to write it either way; just say which before I start.
ποΈ Additional Information
Two things I want to pre-empt, since I checked both:
traceloop-sdkdoes ship@tool/@atool, which create a{name}.toolspan and do setgen_ai.tool.nameβ decorators/base.py#L203-L211. That's manual instrumentation of my own functions; it doesn't hook CrewAI's tool execution, so it can't cover tools CrewAI invokes internally. This request is about the auto-instrumentation path.- This isn't a general semconv complaint. The agent and LLM spans use
gen_ai.*extensively βGEN_AI_PROVIDER_NAME,GEN_AI_OPERATION_NAME,GEN_AI_AGENT_NAME,GEN_AI_INPUT_MESSAGES/OUTPUT_MESSAGESβ and the package shipstest_semconv_compliance.py. The gap is specifically tool execution.
Possibly related: #4397 lists CrewAI among the unwired packages in the conformance contract, pending cassettes. Glad to help wire a CrewAI cassette β I have a working harness.
Design note for whoever implements this: gen_ai.tool.call.id won't be available on CrewAI regardless, since CrewAI doesn't surface the model's tool call id to BaseTool.run (crewAIInc/crewAI#7178, proposed fix in PR #7189 β open, not merged).
π Have you spent some time to check if this feature request has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
Yes I am willing to submit a PR!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up β it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/opentelemetry-instrumentation-crewai/opentelemetry/instrumentation/crewai/instrumentation.py, where Crew.kickoff, Agent.execute_task, Task.execute_sync, and LLM.call are wrapped, and inspect crewai_span_attributes.py for existing tool data. Compare the requested BaseTool.run span with RFC #3460 and the upstream execute_tool convention before choosing the span shape. Update the relevant instrumentation tests, including test_semconv_compliance.py, so tool invocations expose the agreed attributes and content capture behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100