Output tokens logits
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Hello,
Unless I'm mistaken, I don't see any option in the translator translate_batch or generate_tokens functions to output the logits/probabilities of the generated tokens. However, this computation must be done internally before sampling the tokens, right ?
This would be very useful in order to have more control over the generation, for example for influencing the choice of specific tokens. Is there a particular reason why this is not exposed ? Could this be easily added, or am I missing something? This could for example take the form of a return_logits argument to the mentioned functions.
Thanks,
Simon
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the translator's translate_batch and generate_tokens entry points to understand where generated-token logits or probabilities are available before sampling. Define the return format and behavior for an opt-in return_logits argument, then verify that the API exposes the requested values without changing default generation behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100