apache / apache/flink-agents

[Feature] Track token usage metrics for embedding models

Open
#858 2 comments 0 reactions 0 assignees View on GitHub
feature priority/major
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 9h
Merged PRs (30d)
49

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.

### Description

Embedding model calls currently do not consistently report token usage metrics.

Chat models already have a token accounting path: provider usage is attached to the chat response and later recorded as model-level token metrics. Embedding models return only vectors, so provider usage returned by OpenAI-compatible or DashScope-style embedding APIs can be dropped before it reaches the metrics layer.

Affected paths include:

- Direct Java or Python embedding model calls.
- Vector store and RAG paths that auto-generate embeddings during `add`, `update`, or `query`.
- Cross-language resource paths where the wrapper may receive an action metric group but the provider-side embedding resource performs the actual request.

This makes it harder to validate and compare embedding model cost/usage, especially when a job mixes chat, embedding, and vector store operations. Embedding metrics do not need `completionTokens`, but should expose input-side token usage, for example `promptTokens` and `totalTokens`, under the same model/provider metric dimensions used by chat metrics where possible.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing chat-model token accounting path, then follow embedding provider usage through direct Java and Python calls and the vector-store/RAG add, update, and query paths. Check cross-language resource wrappers and verify that embedding requests expose input-side promptTokens and totalTokens under the established model/provider metric dimensions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.