ContextLab / ContextLab/llm-geometry

Architecture Explorer: expand model support beyond the curated list

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Context

Feature 004 removed the Architecture Explorer's free-text "any open-weights HuggingFace id" input and replaced it with a curated dropdown (FR-412). That was the honest move: the static build's live path depends on a community **ONNX** export existing for the model, and most HF repos have none — so the old input promised something the deployed demo could not deliver.

This issue tracks doing it properly.

## Options to evaluate

1. **Grow the curated list.** Cheap and safe. Each entry needs a verified ONNX export (`onnx-community/...`) plus a pinned revision, and needs to fit under the parameter ceiling (`ARCH_MAX_PARAMS`).
2. **Automatic filtering.** Query the HF API for models that (a) carry an open-weights license, (b) have an ONNX export with the file layout transformers.js expects, and (c) are under the size gate — then offer *those* as a searchable list. This keeps the promise honest because the filter is the promise.

## Constraints worth remembering

- ONNX exports do **not** expose attention weights or hidden states, so live per-prompt *tracing* of an arbitrary model in the browser is not possible even when generation works. Any expansion must be explicit about which capabilities a given model gets (generate / tokenize / weight-inspect / trace).
- The full-stack (Python) path can trace anything HF can load; only the static build is constrained. Copy must not blur the two.

## Acceptance

- The picker's affordance matches exactly what each listed model can do.
- No model appears that the runtime cannot actually load.
- Red-team pass finds no claim contradicted by behavior.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.