finos / finos/fluxnova-plugins

[Feature] Add LLM token usage monitoring

Open
#49 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3
Forks
7
Avg merge
5d 4h
Merged PRs (30d)
3

Description

### Description of Problem

When I run LLM workflows, I want to see how many tokens each request uses, so I can control costs and catch problems early.

Right now there is no way to see token usage. The only option is the provider's billing page, which is delayed and only shows account totals. That means I can't:

- Tell which feature, user, or workflow is spending the money
- Catch a runaway loop before it runs up a big bill
- Predict costs when scaling up or switching models

### Potential Solutions

**Proposed: built-in token tracking**

- Record input/output tokens for each request (use the provider's response data, or estimate if not available)
- Group totals by model, session, and tags (user, feature, etc.)
- Show the data in logs and a metrics endpoint
- Add optional alerts when usage passes a set limit
*Benefits:* real-time cost visibility, spend tied to specific workflows, early warning on runaway usage.

*Drawbacks:* small overhead per request; estimates may be off for some providers; more config to maintain.

**Alternatives considered**

- **External proxy/gateway:** no code changes, but adds infrastructure and latency, and loses in-app context.
- **Manual logging:** works, but everyone builds it differently and there's no shared standard.
- **Provider billing dashboards:** free, but delayed and account-level only.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, providers, or metrics entry points. Start by locating where provider responses are handled and how logs, metrics, tags, and configuration are exposed; then clarify which providers and alerting behavior are in scope. Done should include per-request input/output usage, grouped totals, logs, a metrics endpoint, and optional limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.