HarperFast / HarperFast/harper

models: no metadata accessor on the facade — consumers must burn an embed call to learn dimensionality

Open
#1,757 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Problem

The `models` facade (#510) exposes `embed`/`generate`/`generateStream` plus registration methods, but no way to ask *about* a configured model. Consumers building vector storage on top of `models.embed()` need, at minimum, the vector **dimensionality** — for the vector column shape / HNSW config, for stale-embedding detection, and for re-embed bookkeeping when the underlying model changes.

Today the only options are hardcoding dims per model or burning an embed call and measuring the result:

```js
const dims = (await models.embed('probe'))[0].length;
```

That's a real inference call (cost, latency, an `hdb_model_calls` analytics row) to read a static property. A downstream consumer hit exactly this while integrating a config-registered embedding backend (heskew/harper-fabric-embeddings#3 — see the review discussion about dims + re-embed planning).

## Sketch

Optional metadata on the backend contract, surfaced through the facade with the same logical-name resolution as calls:

```ts
interface ModelBackend {
// ...
describe?(): MaybePromise;
}

interface ModelMetadata {
/** Wire model id the backend is configured with, if known. */
model?: string;
/** Embedding dimensionality, if known. */
dimensions?: number;
/** Context window in tokens, if known. */
contextTokens?: number;
}

// facade — resolves logicalName via the router like embed()/generate() do
models.describe(kind: 'embedding' | 'generative', logicalName?: string):
Promise<{ backend: string; capabilities: ModelCapabilities; metadata?: ModelMetadata }>
```

`defineBackend` would pass `describe` through from the spec.

## Design questions (input wanted before implementation)

- **Sync vs async.** For lazy-loading in-process backends (a GGUF engine that fast-boots and loads on first use), dims are unknown until the model is loaded. Async `describe()` lets such a backend await readiness; the alternative (sync, `undefined` until ready) makes the accessor unreliable exactly when consumers need it (startup/schema time). Async seems right; it's a cold-path call.
- **Shape.** A dedicated `describe()` vs widening `capabilities()` — capabilities is a boolean feature matrix consumed by the router on the hot path; metadata is different in kind and optional, so a separate method seems cleaner.
- **Built-in coverage.** ollama can report dims only after a probe call or from a model catalog; openai dims are model-table knowledge. Whether built-ins ship `describe()` in the first cut or it starts as a pass-through for custom backends (#1325/#1471) is scoping.

Related: #783 (log/surface registered backends at startup — same underlying need for backend introspection, operator-facing rather than consumer-facing), #1235 (models subsystem audit).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Start by reading the models facade, the router resolution used by embed() and generate(), and the defineBackend contract. Resolve the sync-versus-async, metadata shape, and built-in coverage questions before implementation; the work is done when the agreed describe() behavior is consistently exposed for registered backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.