NVIDIA / NVIDIA/TransformerEngine
cuDNN to accept actual seqlens in addition to cumulative seqlens
Open
@KshitijLakhani is already working on this.
Since Jul 16, 2026.
attention
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 831
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 65
Description
https://github.com/NVIDIA/TransformerEngine/pull/3186/changes recently added support for TE to pass cu_seqlens* tensors directly to cuDNN.
It'd be nice if cuDNN could also accept "actual seqlens" because TE-JAX has to launch a kernel for it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.