huggingface / huggingface/evaluate
is it possible to evaluate a proprietary model without uploading it to HF
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
I really like HF 'evaluate'.
However, if my customer is not allowed to publicly share his/her model, how can they still use HF 'evaluate' module?
In the code, I see 'model_id' always referencing a model from a public HF model hub (eg. "lvwerra/distilbert-imdb").
Is there a way to implement a Bring-Your-Own-Model capability?
Contributor guide
Assessment
This issue has not been assessed yet.