apache / apache/texera

HuggingFace inference operator: provider cost ranking is a hardcoded static list

Open
#7,206 3 comments 0 reactions 1 assignee Claimed by @juliethecao View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

`PROVIDER_COST_PRIORITY` is a hardcoded cheapest-first list; new providers land unranked at the end and the ordering goes stale.

### Context
No public HF API exposes per-model/provider pricing (verify at implementation time), so live pricing isn't practical.

### Proposed change
Either:
- (a) delegate provider selection to the HF router model-level endpoint (`router.huggingface.co/v1/chat/completions` with the Hub ID, HF picks the provider per the user's account preference; chat tasks only), or
- (b) add an optional "preferred providers" operator property prepended to the static ranking.

### Related
Best done after the provider-memoization task (#7202), the same fallback loop, and memoization is the bigger practical win.

### Task Type

- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [x] Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.