HuggingFace inference operator: provider cost ranking is a hardcoded static list
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
`PROVIDER_COST_PRIORITY` is a hardcoded cheapest-first list; new providers land unranked at the end and the ordering goes stale.
### Context
No public HF API exposes per-model/provider pricing (verify at implementation time), so live pricing isn't practical.
### Proposed change
Either:
- (a) delegate provider selection to the HF router model-level endpoint (`router.huggingface.co/v1/chat/completions` with the Hub ID, HF picks the provider per the user's account preference; chat tasks only), or
- (b) add an optional "preferred providers" operator property prepended to the static ranking.
### Related
Best done after the provider-memoization task (#7202), the same fallback loop, and memoization is the bigger practical win.
### Task Type
- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [x] Other
Contributor guide
Assessment
This issue has not been assessed yet.