memory leakage problem when loss.backward()
- Dominant language
- Python
- Stars
- 617
- Forks
- 57
- PR merge metrics
- No merged PRs in 30d
Description
Hi.
How can i solve memory leakage problem on loss.backward?
Actually my code is a bit complex, which makes it hard to provide whole.
The main memory burden comes from here. Exactly after the execution of loss.backward(). The memory increases for every iteration, which leads to OOM.
```python
loss = bce_extended(logits, y).sum()
with backpack(BatchGrad()):
if real_sample:
loss.backward(inputs = list(model.parameters()))
```
I also tried ```with disable():``` which prevent the memory leak problem. However, it cannot be implemented with ```with backpack(BatchGrad()):``` when i want to get the per-sample gradient.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.