crewAIInc / crewAIInc/crewAI

[BUG] crew.usage_metrics multiplies token usage by the number of agents sharing an LLM instance

Open
#7,259 1 comment 0 reactions 1 assignee View on GitHub

@Vidit-Ostwal is already working on this.

Since Sep 4, 2026.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

Crew.calculate_usage_metrics() loops over self.agents and adds agent.llm.get_token_usage_summary() for each one. Those counters are cumulative for the lifetime of the LLM instance, so when several agents share the same LLM object its total is added once per agent and crew.usage_metrics reports N times the real token usage.

Sharing a single LLM instance across agents is the usual way to write a crew, so this affects ordinary usage rather than an edge case. Nothing errors — the reported number is simply wrong, and the inflation grows as agents are added.

get_token_usage_summary() documents this explicitly:

The counters are cumulative for the lifetime of this instance: they grow across every call made through it, including calls issued by different agents sharing the instance.

Steps to Reproduce
  1. Create one LLM instance and give it to two agents.
  2. Run the crew.
  3. Compare crew.usage_metrics.total_tokens with llm.get_token_usage_summary().total_tokens.
Expected behavior

crew.usage_metrics should report the crew's actual token usage. Each distinct LLM instance should contribute its totals exactly once, regardless of how many agents hold a reference to it. Agents with separate LLM instances should still sum normally.

Screenshots/Code snippets
llm = LLM(model="ollama/qwen2.5:14b")
a1 = Agent(role="A", goal="Answer in one word.", backstory="B", llm=llm)
a2 = Agent(role="B", goal="Answer in one word.", backstory="B", llm=llm)  # same instance
crew = Crew(agents=[a1, a2], tasks=[t1, t2])
crew.kickoff()

llm.get_token_usage_summary().total_tokens   # 155
crew.usage_metrics.total_tokens              # 310
agents sharing one LLM instance: True
LLM instance actual total  : 155
crew.usage_metrics reports : 310
ratio                      : 2.00x

The inflation is linear in the number of agents sharing the instance. Stubbing the summary at a known 100 tokens and varying the agent count:

1 agent(s)  sharing 1 LLM (real 100) -> reported  100  (1x)
2 agent(s)  sharing 1 LLM (real 100) -> reported  200  (2x)
3 agent(s)  sharing 1 LLM (real 100) -> reported  300  (3x)
5 agent(s)  sharing 1 LLM (real 100) -> reported  500  (5x)
Operating System

Other (specify in additional context)

Python Version

3.12

crewAI Version

1.15.18

crewAI Tools Version

1.15.18

Virtual Environment

Venv

Evidence
  • lib/crewai/src/crewai/crew.py:2208-2216 — the loop adds agent.llm.get_token_usage_summary() once per agent, with no de-duplication of the underlying instance.
  • lib/crewai/src/crewai/llms/base_llm.py:1004get_token_usage_summary() returns lifetime totals for the instance and its docstring notes those include calls from other agents sharing it.
  • The same applies to manager_agent.llm at crew.py:2222-2225 when the manager shares an instance with the agents.

Reproduced both with a live crew and deterministically by stubbing the summary, so the ratio does not depend on model behaviour.

Possible Solution

Track which LLM instances have already contributed, by object identity, and skip repeats:

counted_llms: set[int] = set()

for agent in self.agents:
    if isinstance(agent.llm, BaseLLM):
        if id(agent.llm) in counted_llms:
            continue
        counted_llms.add(id(agent.llm))
        ...

with the same identity check applied to the manager agent's LLM. id() rather than model name matters here: two agents may legitimately hold separate LLM(model="...") instances with the same model, and those must still sum.

Additional context

Related but distinct: #4934 (open since 2026-03-18) fixes a different double-count in this same function, where the manager agent is counted twice because _token_process and the LLM summary are added in two independent if blocks rather than mutually exclusive branches. That change is still needed after this one; the two address different mechanisms. I've deliberately not touched the if/else restructure that PR proposes.

My OS is macOS Tahoe 26.5.2 and I'm on Python 3.13.13, neither of which is in the dropdowns. Running from a source checkout of main at commit 92eb5f9.

This issue was written with AI assistance and should carry the llm-generated label per CONTRIBUTING.md. I can't apply labels myself — could a maintainer add it?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.