Details about the Slide-LoRA

Hello, I’ve read the paper and am trying to replicate the Slide-LoRA module in my model. However, I'm not entirely clear on whether the gating network operates at the token level or the sequence level. Could you kindly provide the insights or point out the part of the code where the Slide-LoRA module and the gating network are specifically implemented?

No file or test is named. Search the repository for the Slide-LoRA module and its gating network, then trace how inputs are passed through them. Done means documenting the implementation location and clearly stating whether gating operates per token or per sequence.

Written by the indexing model from the issue text.