Pytorch workflow Logits Processor problem
Open
@hchings is already working on this.
Since Jul 22, 2025.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
trtllm version: 0.20.0
GPU: 4090
class CombinedLogitsProcessor(LogitsProcessor):
def __init__(self, stra) -> None:
self.stra = stra
self.step = 0
def __call__(self, req_id: int, logits: torch.Tensor,
token_ids: List[List[int]], stream_ptr: int,
client_id: Optional[int]):
print(self.step, len(token_ids[0]))
self.step += 1
I wrote this processor, when disable_overlap_scheduler is False(default value),
It will always output the same length for step 0 and 1, and the current logit strategy will not cause effect to the next token, but to the next next token.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.