Relative positional encoding as used by RETRO

Hi,

This is not an issue but a question about a detail in the RETRO paper.
I'm currently looking in to the paper and would like to better understand how the relative positional encoding is done.

The paper is very cryptic about this, saying (section B.1.2):

> Positional logits are obtained as a linear transform of a cosine vector computed from (𝑑(𝑖, 𝑖′))𝑖,𝑖′.

Could anyone give more insight in to what this actually means? How are the positional logits calculated from the distances?

The paper later on provides JAX pseudo code for the chunked cross attention (Section B.1.3, Listing 1). Here the relative positional encoding is described as "Classical relative positional encodings". To me, classical relative positional encodings, is how positional embeddings are added in the original Transformer paper (Attention is all you need). However, here we are not interested in vector embeddings for positional encoding, but scalar values we can add to attention logits. So it confuses me what the above quoted statement actually means.

Many thanks in advance for your response!

(And thanks for the great research! :) )