Relative positions in RoPE embeddings

Hi,
I was going through your code to know how you calculated the RoPE embeddings and need a clarification

In assigning a relative position to a newly generated token, the base reference is taken as the end of the prompt input
https://github.com/abertsch72/unlimiformer/blob/232fc235706c304667f7a671cca2203d4625eaa1/src/unlimiformer.py#L1084C10-L1084C10

In assigning a relative position to the retrieved key indices the relative position is taken as the start of the prompt input
https://github.com/abertsch72/unlimiformer/blob/232fc235706c304667f7a671cca2203d4625eaa1/src/unlimiformer.py#L1123

Then would it not be the case that the current hidden state gives more attention to the tokens somewhere in the middle of the prompt and then decays both to the right and left?

Thank you
Ashwin Ramachandran

該 issue 指向 src/unlimiformer.py 中計算 RoPE embeddings 的第 1084 行和第 1123 行。首先理解新 token 與擷取到的 key 索引之間的相對位置邏輯。檢查 attention 機制，確認 attention 是否對稱衰減。在長距離輸入上執行測試，以驗證其行為。

由索引模型根據 Issue 內容生成。