ByteDance-Seed / ByteDance-Seed/FlexPrefill

about query aware index

Open
#15 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
172
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Hello, in the code provided, the query aware attention index is concat with vertical-slash index(https://github.com/ByteDance-Seed/FlexPrefill/blob/main/flex_prefill/ops/flex_prefill_attention.py#L1778). This conflicts to the paper's Algorithm 1. In Algorithm 1, vertical-slash index is not used in query aware attention. I wonder this may affect performance of FlexPrefill?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading flex_prefill/ops/flex_prefill_attention.py around line 1778 and compare the query-aware attention index construction with Algorithm 1 in the paper. Measure whether including the vertical-slash index changes FlexPrefill performance, then determine whether the implementation or the paper needs correction and record the evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.