huggingface / huggingface/lighteval
[INFO] Document or remove `logits` from the `ModelResponse` class
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 555
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 1
Description
@NathanHB I was exploring the ModelResponse class post refacto, and I noticed we have both logits (undocumented) and logprobs (documented) in the class. Logits is used for some classes (transforemrs) and logprob for others - I believe we should have a single class for both as they are usually used interchangeably and would allow us to have a single system. If we actually want to manage them differently, we can but it needs more documentation.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.