HarperFast / HarperFast/harper
models: no metadata accessor on the facade — consumers must burn an embed call to learn dimensionality
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Problem
The `models` facade (#510) exposes `embed`/`generate`/`generateStream` plus registration methods, but no way to ask *about* a configured model. Consumers building vector storage on top of `models.embed()` need, at minimum, the vector **dimensionality** — for the vector column shape / HNSW config, for stale-embedding detection, and for re-embed bookkeeping when the underlying model changes.
Today the only options are hardcoding dims per model or burning an embed call and measuring the result:
```js
const dims = (await models.embed('probe'))[0].length;
```
That's a real inference call (cost, latency, an `hdb_model_calls` analytics row) to read a static property. A downstream consumer hit exactly this while integrating a config-registered embedding backend (heskew/harper-fabric-embeddings#3 — see the review discussion about dims + re-embed planning).
## Sketch
Optional metadata on the backend contract, surfaced through the facade with the same logical-name resolution as calls:
```ts
interface ModelBackend {
// ...
describe?(): MaybePromise;
}
interface ModelMetadata {
/** Wire model id the backend is configured with, if known. */
model?: string;
/** Embedding dimensionality, if known. */
dimensions?: number;
/** Context window in tokens, if known. */
contextTokens?: number;
}
// facade — resolves logicalName via the router like embed()/generate() do
models.describe(kind: 'embedding' | 'generative', logicalName?: string):
Promise<{ backend: string; capabilities: ModelCapabilities; metadata?: ModelMetadata }>
```
`defineBackend` would pass `describe` through from the spec.
## Design questions (input wanted before implementation)
- **Sync vs async.** For lazy-loading in-process backends (a GGUF engine that fast-boots and loads on first use), dims are unknown until the model is loaded. Async `describe()` lets such a backend await readiness; the alternative (sync, `undefined` until ready) makes the accessor unreliable exactly when consumers need it (startup/schema time). Async seems right; it's a cold-path call.
- **Shape.** A dedicated `describe()` vs widening `capabilities()` — capabilities is a boolean feature matrix consumed by the router on the hot path; metadata is different in kind and optional, so a separate method seems cleaner.
- **Built-in coverage.** ollama can report dims only after a probe call or from a model catalog; openai dims are model-table knowledge. Whether built-ins ship `describe()` in the first cut or it starts as a pass-through for custom backends (#1325/#1471) is scoping.
Related: #783 (log/surface registered backends at startup — same underlying need for backend introspection, operator-facing rather than consumer-facing), #1235 (models subsystem audit).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start by reading the models facade, the router resolution used by embed() and generate(), and the defineBackend contract. Resolve the sync-versus-async, metadata shape, and built-in coverage questions before implementation; the work is done when the agreed describe() behavior is consistently exposed for registered backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100