ContextLab / ContextLab/orchestrator
HuggingFace Inference API provider
- Dominant language
- Python
- Stars
- 3
- Forks
- 2
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
## Context
The provider set is Dartmouth Chat + HuggingFace only (decided 2026-08-21; ADR 0001). HuggingFace support means the **hosted Inference API**, not the local transformers inference in `integrations/huggingface_model.py` / `lazy_huggingface_model.py` — those stay in the frozen layer and retire under #430.
## What to build
A `HuggingFaceProvider` in the canonical layer (`models/providers/`), speaking the Inference API over HTTP with `aiohttp` (already a core dependency — same shape as `DartmouthProvider`, no new core deps, no provider extra required unless the SDK is used).
- Text-generation against a small set of served models, selected from the live catalog rather than a hard-coded list (two rounds of hard-coded Anthropic ids were wrong; do not repeat that here).
- `HF_TOKEN` from the environment or `~/.orchestrator/.env`, same as other credentials.
- Free-tier vs paid distinction must fail closed: a model the token cannot serve must raise, not silently route elsewhere. Model fallback across providers does not exist (ADR 0001).
- Registered in the normal model registry when a credential is present, so pipelines can select the models without touching the provider directly.
## Acceptance criteria
- [ ] Contract tests (hermetic, recorded/mocked HTTP boundary is fine at this layer) covering: request shape, auth header, error mapping (401/403/402/429/5xx), model-unavailable, and the registry integration.
- [ ] A `live-huggingface` job in `live-tests.yml`, mirroring `live-dartmouth`: hard-fails when `HF_TOKEN` is missing under `ORCHESTRATOR_REQUIRE_LIVE=1`, and fails when every test skips.
- [ ] Live tests exercise at least one real served model end-to-end: `generate()` returns non-empty content, `models.list()`/discovery works, `health_check()` works.
- [ ] Reasoning-model behaviour handled: a response that spends its token budget on reasoning and returns empty content must not be reported as a successful empty answer (learned from the Dartmouth work — see #432 comments).
- [ ] README provider table flips HuggingFace to **supported** only after the live job passes remotely.
## Out of scope
- Local transformers/torch inference (frozen; retires with #430).
- `text-generation-inference` self-hosting, Inference Endpoints (dedicated), and any SDK dependency in core.
Related: #429 (adapter-layer consolidation — this provider lands in `models/providers/`), #430 (retirement of the frozen layer), ADR 0001 provider policy.
Contributor guide
Assessment
This issue has not been assessed yet.