huggingface / huggingface/evaluate
Allow for specify coda device in perplexity evaluation
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
The current implementation of `perplexity.compute` does not allow to provide a specific coda device (e.g. `cuda:1` instead of just `cuda`.
https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/metrics/perplexity/perplexity.py#L108
Should probably be changed to something along the lines of:
```python
assert device in ["gpu", "cpu", "cuda"] or re.search(r'^cuda:[0-9]+$', device), "device should be either gpu or cpu."
```
Contributor guide
Assessment
This issue has not been assessed yet.