huggingface / huggingface/evaluate
Perplexity for quantized models
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Hi, If I intend to compute the perplexity for a quantized model, what parameters should I pass to the .compute() function?
For example,
>>> perplexity = load("perplexity", module_type="metric")
>>> good_result = perplexity.compute(predictions=SENTENCES,model_id=MODEL_NAME,add_start_token=False)
Much thx!
Contributor guide
Assessment
This issue has not been assessed yet.