huggingface / huggingface/peft
`get_peft_model` and `add_adapter` do not preserve eval mode for newly injected adapter modules
- Dominant language
- Python
- Stars
- 21.7k
- Forks
- 2.5k
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 59
Description
### System Info
- PEFT: `0.21.0` (`main`, commit `514b9e6d`)
- Transformers: `5.3.0`
- Python: `3.12.3`
### Who can help?
@BenjaminBossan
### Reproduction
I noticed that injecting a LoRA adapter into an already-evaluated model can silently enable adapter dropout.
```python
import torch
from torch import nn
from peft import LoraConfig, get_peft_model
class Tiny(nn.Module):
def __init__(self):
super().__init__()
self.lin = nn.Linear(4, 4, bias=False)
def forward(self, x):
return self.lin(x)
config = LoraConfig(
target_modules=["lin"],
r=2,
lora_alpha=2,
lora_dropout=0.5,
init_lora_weights=False,
)
model = get_peft_model(Tiny().eval(), config)
layer = model.base_model.model.lin
with torch.no_grad():
layer.lora_A["default"].weight.fill_(1.0)
layer.lora_B["default"].weight.fill_(1.0)
x = torch.ones(4, 4)
with torch.no_grad():
output_1 = model(x)
output_2 = model(x)
print(model.training) # True
print(layer.lora_dropout["default"].training) # True
print(torch.equal(output_1, output_2)) # False
```
The base model was in evaluation mode before adapter injection, so I expected the resulting PEFT model and adapter modules to remain in evaluation mode. Instead, the newly created LoRA dropout module stays in training mode, making repeated inference nondeterministic.
The same happens when adding an adapter to an existing evaluated PEFT model:
```python
model = get_peft_model(
Tiny(),
LoraConfig(target_modules=["lin"], lora_dropout=0.0, init_lora_weights=False),
).eval()
model.add_adapter("other", config)
model.set_adapter("other")
print(model.training) # False
print(model.base_model.model.lin.training) # False
print(model.base_model.model.lin.lora_dropout["other"].training) # True
```
In this case, the parent layer is correctly in evaluation mode, but the newly added adapter dropout is not.
### Expected behavior
Expected behavior is that newly injected adapter modules inherit the current training/evaluation state of the model or parent module. This seems different from the existing adapter state-restoration issue in #3507 because the problem occurs during adapter injection rather than `disable_adapter()` context handling.
I’d be happy to open a PR with a shared fix and regression tests for both initial injection and `add_adapter`. Before doing that, would you prefer preserving the full incoming PEFT model’s mode, or explicitly inheriting the training state from each parent module when new adapter modules are created?
Contributor guide
Research direction
Start at the get_peft_model and add_adapter entry points and trace how newly created LoRA adapter modules are attached to their parent modules. Add regression tests for both initial injection and add_adapter, verifying that adapter dropout inherits the existing training or evaluation state and repeated evaluation is deterministic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100