huggingface / huggingface/lighteval

[INFO] Document or remove `logits` from the `ModelResponse` class

Open
#881 0 comments 0 reactions 1 assignee Claimed by @NathanHB View on GitHub
Dominant language
Python
Stars
2.5k
Forks
555
Avg merge
1d 6h
Merged PRs (30d)
1

Description

@NathanHB I was exploring the ModelResponse class post refacto, and I noticed we have both logits (undocumented) and logprobs (documented) in the class. Logits is used for some classes (transforemrs) and logprob for others - I believe we should have a single class for both as they are usually used interchangeably and would allow us to have a single system. If we actually want to manage them differently, we can but it needs more documentation.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.