[Feature] Track token usage metrics for embedding models
- Dominant language
- Java
- Stars
- 452
- Forks
- 167
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 49
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.
### Description
Embedding model calls currently do not consistently report token usage metrics.
Chat models already have a token accounting path: provider usage is attached to the chat response and later recorded as model-level token metrics. Embedding models return only vectors, so provider usage returned by OpenAI-compatible or DashScope-style embedding APIs can be dropped before it reaches the metrics layer.
Affected paths include:
- Direct Java or Python embedding model calls.
- Vector store and RAG paths that auto-generate embeddings during `add`, `update`, or `query`.
- Cross-language resource paths where the wrapper may receive an action metric group but the provider-side embedding resource performs the actual request.
This makes it harder to validate and compare embedding model cost/usage, especially when a job mixes chat, embedding, and vector store operations. Embedding metrics do not need `completionTokens`, but should expose input-side token usage, for example `promptTokens` and `totalTokens`, under the same model/provider metric dimensions used by chat metrics where possible.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start by tracing the existing chat-model token accounting path, then follow embedding provider usage through direct Java and Python calls and the vector-store/RAG add, update, and query paths. Check cross-language resource wrappers and verify that embedding requests expose input-side promptTokens and totalTokens under the established model/provider metric dimensions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100