HarperFast / HarperFast/harper

[Models] Models#embed() discards backend token usage — /v1/embeddings usage always reports zero

Open
#1,882 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

Surfaced by @kriszyp reviewing #1616.

`/v1/embeddings` always returns `usage: { prompt_tokens: 0, total_tokens: 0 }`. The v1 mapper (`toEmbedResponse` in `resources/models/v1/translation.ts`) accepts an optional `usage` argument and maps `embeddingTokens`/`promptTokens` correctly when given one, but the caller can't supply it:

Root cause is upstream in the Models facade. `Models#embed()` (`resources/models/Models.ts`) returns `Promise` — it captures `result.usage` internally for its own `#record()`/billing bookkeeping, then returns only `result.output`, discarding usage before the caller sees it. So `V1Embeddings` has no way to recover real token counts.

This is an asymmetry in the facade, not something introduced by the gateway: `generate()` exposes `usage` on its `GenerateResult` and `/v1/chat/completions` wires it through correctly; only the embed path drops it.

Impact: any OpenAI client that inspects `usage.total_tokens` on this endpoint (cost dashboards, budget guards) always sees zero. Low severity — the numbers are captured for internal accounting, just not surfaced on the wire.

Fix direction: expose usage from `Models#embed()` (e.g. return an `EmbedResult` with `{ output, usage }`, or an overload) and wire it into `toEmbedResponse`. This is a public facade signature change (`scope.models.embed()`), so it's deliberately out of scope for the #631 gateway PR and tracked here instead.

Context: #631 (Phase 4 of #510).

🤖 Filed with [Claude Code](https://claude.com/claude-code) on behalf of @heskew from #1616 review

Contributor guide

Open the contributing guide

Research direction

Start in resources/models/Models.ts to trace Models#embed() and compare its result handling with generate(), then inspect toEmbedResponse in resources/models/v1/translation.ts. The work is done when backend embedding usage reaches the v1 response and /v1/embeddings reports the real prompt and total token counts instead of zero.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, typescript
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.