huggingface / huggingface/evaluate
perplexity.compute - Add args to pass model and tokenizer objects instead of `model_id`
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I am new to the project, so I hope my request is a sensible one.
It looks like when we use the compute method for Perplexity, we can only specify a model through its `model_id`, which is used also to load the tokenizer.
Is there any particular reason why it is not possible to pass as arguments to the method the model object and the tokenizer object themselves instead of the `model_id`? I think that could help, for example when the tokenizer has some user-added tokens or when the model to be used is fine-tuned (e.g., currently, a PeftModel could not be used, but passing the model object itself that should not be a limitation anymore).
Thanks for your answer and for your amazing work.
Contributor guide
Assessment
This issue has not been assessed yet.