NVIDIA / NVIDIA/recsys-examples
[QST] Timeline for "native page tables in decode attention" for SID-GR — variable-length online throughput gap
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 322
- Forks
- 83
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 8
Description
README §10 (ContextKV memory strategy) lists "eventually support native page tables in decode attention".
On variable-length online input (128–256 tok, beam=50), SID-GR trails SGLang by ~2x - 3x on L20, with decode collapsing to all batch=1. This gap can't really close until paged decode attention lands — the interim steps (multi context buckets, page-backed ContextKV) only mitigate.
Is there a rough timeline / priority for native paged decode attention? Are the interim steps planned sooner?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with README §10, especially the ContextKV memory strategy and its note about native page tables in decode attention. Review the reported variable-length online benchmark conditions and determine whether the project has documented plans for paged decode attention, multi-context buckets, or page-backed ContextKV; done means providing a confirmed priority or timeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100