Simple local token tracking for Pipelines

Idea

Discussing with Robin, it turned out that it could be useful to have a simple component/utility
to make tokens usage visible and organized.

While tracers provide this, they typically require either sending data to an external service
or running a local backend (e.g., Docker).

Ideal features

• track token usage per component in a Pipeline and per model name
• local execution
• no backend or lightweight one
• integrate naturally with Pipelines (not a component that must be explicitly connected to many others)

Non-goals: persistence, visualization, cost estimation

Possible design

This could follow a pattern similar to LangfuseConnector and others.

(might be hard to implement or go against the framework design)

pipe = Pipeline()
pipe.add_component("token_tracker", TokenTracker())
pipe.add_component("prompt_builder", ChatPromptBuilder())
pipe.add_component("llm", OpenAIChatGenerator())

pipe.connect("prompt_builder.prompt", "llm.messages")

messages = [ChatMessage.from_user("Tell me about {{location}}")]

response = pipe.run(
    data={
        "prompt_builder": {
            "template_variables": {"location": "Berlin"},
            "tmpelate": messages,
        },
    },
)
print(response["token_tracker"])
# info on token usage

An alternative approach would be closer to a tracer (like Haystack-Datadog integration), but this would probably need
some kind of backend. Still, it could be useful to explore this direction.

Future work

If this feature is relevant, a good first step would be to experiment with existing supported tracers (those that allow execution). They may already satisfy most of the requirements, or at least provide an inspiration for a lightweight solution.

@deep-rloebbert feel free to jump in: corrections, details, and examples are welcome.

👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!

Start by experimenting with the existing supported tracers that allow local execution, then compare their behavior with the LangfuseConnector pattern and the Haystack-Datadog tracing integration. The issue does not identify files or tests, and completion criteria remain exploratory: determine whether a lightweight local token-tracking approach fits Pipelines without persistence, visualization, or cost estimation.