alipay / alipay/PainlessInferenceAcceleration
modeling_qwen attention not use multi branch position ids & attention_mask
Offen
- Vorherrschende Sprache
- Python
- Sterne
- 370
- Forks
- 23
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
I reviewed the code of modeling_qwen.py, and I noticed that, within the lookahead process, the draft_ids matched from the TrieTree are such that the attention_mask and position ids associated with these draft_ids are not being utilized in the attention mechanism. This, I believe, might be an implementation error. Could you please point out where my understanding is incorrect?
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.