Whisper Model Has Inaccessible Logits/Log Probs
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Description:
Hello, I've been tinkering with CTranslate2 with whisper converted to ct2. Using model.generate with return_logits_vocab=True and return_scores=True I'm unable to access or print out the logits entirely. For example:
result = model.generate(..., return_logits_vocab=True, return_scores=True)
print(result[0].logits[0][0])
# Output: [cpu:0 float32 storage viewed as ]
print(result[0].logits[0][0].__array_interface__)
# Output: {'shape': (), 'typestr': '<f4', 'data': (0, False), 'version': 3}
This issue is reproducible with any input, even though the sequences are transcribed correctly.
Expected Behavior:
I would expect result[0].logits to contain accessible tensors with the logit values or log probs for each token generated.
Actual Behavior:
The logits appear to be empty or not fully accessible, as indicated by the output above.
This is super strange to me because result[0].scores[0] is completely accessible and prints out exactly as expected and I'd have to imagine that the scores attribute is just some mean or derivative of all of the log probs/logits for each token generated by CTranslate2. Which leads me to believe that this is some sort of bug rather than intentional behavior to hide the log probs.
Additional Details:
-
CTranslate2 Version: 4.5.0
-
I also tried the latest commit (6a3dc63db5608567bf554ca5dbbfdac1f7a2321f) which seemed related to the logits issues. While the commit added the logits property on print WhisperGenerationResult with print(result[0]), even there the issue persists with the same empty logits and everything else working:
logits=[[ [cpu:0 float32 storage viewed as ]], [ [cpu:0 float32 storage viewed as ]], [ [cpu:0 float32 storage viewed as ]], [ [cpu:0 float32 storage viewed as ]], [ [cpu:0 float32 storage viewed as ]]
Environment:
Python 3.11.11
MacOS Sequoia 15.2
Running cpu float32 inference on an M1 Pro Macbook.
Please let me know if you need any more information. If this is indeed a bug (and not intended behavior for the Whisper model), I'm happy to investigate further or contribute a fix.
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Whisper path through the Python model.generate call with return_logits_vocab=True and return_scores=True, then inspect the WhisperGenerationResult logits exposed by that API. Done means result[0].logits contains accessible values for each generated token, while the existing scores behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- api, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100