LoRA is activated even without trigger word

Hi,
I've trained a LoRA using Kohya SS repo.
After the training, I see that the LoRA changes the results of the generation even without using the trigger words. I trained the LoRA using the dataset convention num-repeats_trigger-word (i.e. 20_background, which is my trigger word).
Any idea why? Should I just load it whenever I want to use it? This would slow down inference significantly...

Start by reproducing the reported generation behavior with the trained LoRA, comparing results with and without the trigger word. Review the Kohya SS training setup and the dataset naming convention `20_background`; done means identifying whether this is expected behavior or a reproducible defect and documenting the cause.

Written by the indexing model from the issue text.