HarperFast / HarperFast/harper
[Models] Models#embed() discards backend token usage — /v1/embeddings usage always reports zero
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
Surfaced by @kriszyp reviewing #1616.
`/v1/embeddings` always returns `usage: { prompt_tokens: 0, total_tokens: 0 }`. The v1 mapper (`toEmbedResponse` in `resources/models/v1/translation.ts`) accepts an optional `usage` argument and maps `embeddingTokens`/`promptTokens` correctly when given one, but the caller can't supply it:
Root cause is upstream in the Models facade. `Models#embed()` (`resources/models/Models.ts`) returns `Promise` — it captures `result.usage` internally for its own `#record()`/billing bookkeeping, then returns only `result.output`, discarding usage before the caller sees it. So `V1Embeddings` has no way to recover real token counts.
This is an asymmetry in the facade, not something introduced by the gateway: `generate()` exposes `usage` on its `GenerateResult` and `/v1/chat/completions` wires it through correctly; only the embed path drops it.
Impact: any OpenAI client that inspects `usage.total_tokens` on this endpoint (cost dashboards, budget guards) always sees zero. Low severity — the numbers are captured for internal accounting, just not surfaced on the wire.
Fix direction: expose usage from `Models#embed()` (e.g. return an `EmbedResult` with `{ output, usage }`, or an overload) and wire it into `toEmbedResponse`. This is a public facade signature change (`scope.models.embed()`), so it's deliberately out of scope for the #631 gateway PR and tracked here instead.
Context: #631 (Phase 4 of #510).
🤖 Filed with [Claude Code](https://claude.com/claude-code) on behalf of @heskew from #1616 review
Contributor guide
Research direction
Start in resources/models/Models.ts to trace Models#embed() and compare its result handling with generate(), then inspect toEmbedResponse in resources/models/v1/translation.ts. The work is done when backend embedding usage reaches the v1 response and /v1/embeddings reports the real prompt and total token counts instead of zero.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, typescript
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100