facebookresearch / facebookresearch/fairscale
Error Freezing Weights
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 293
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to freeze weights of linear layers but I get this error:
```
module.weight.requires_grad = not freeze
RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach().
```
and in my case the `module` is `ColumnParallelLinear` whose weight does seem to be a leaf variable:
https://github.com/facebookresearch/fairscale/blob/164cc0f3170b4a3951dd84dda29c3e1504ac4d6e/fairscale/nn/model_parallel/layers.py#L262
so I am not sure why I am getting this error message.
Contributor guide
Research direction
Start in fairscale/nn/model_parallel/layers.py around line 262 and reproduce the reported requires_grad assignment with ColumnParallelLinear. Trace how its weight is created and determine why it is not treated as a leaf variable; done means the cause and the supported freezing behavior are established, with a regression test if the repository provides a suitable test location.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100