f-dangel / f-dangel/backpack

AttributeError: 'Parameter' object has no attribute 'grad_batch'

Open
#270 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
617
Forks
57
PR merge metrics
No merged PRs in 30d

Description

Hi,
I meet some problems when using BackPACK with RNNs.
the decode part is the last several layer of my model,and here is the code. I am interested in its individual gradients ,so extend this modules.
```
class decode(nn.Module):
def __init__(self,args):
super(decode, self).__init__()
self.args = args
self.soc_embedding_size = (((args['grid_size'][0]-4)+1)//2)*self.conv_3x1_depth*5
self.dyn_embedding_size = args['dyn_embedding_size']
self.decoder_size =args['decoder_size']

self.dec_lstm = torch.nn.LSTM(self.soc_embedding_size + self.dyn_embedding_size, self.decoder_size ,batch_first=True)
self.op = torch.nn.Linear(self.decoder_size,2)

def forward(self,enc):

h_dec, _ = self.dec_lstm(enc)

fut_pred = self.op(h_dec)

return fut_pred
```

and I refer to your documentation using the 'use_converter=True'. And I use nn.MSEloss for the loss function.
the part of using backpack is as following:
```
with backpack(BatchGrad()):
loss.backward(
inputs=list(decode.parameters(),retain_graph=True,create_graph=True)
)
```

but when I want to check whether it is working by this code
```
for name, weights in decode.named_parameters():
count=count+1
print(count)
print('name',name)
print('weights',weights.shape)
print(weights.requires_grad)
print(weights.grad_batch.shape)
```

there always has error.
```
Traceback (most recent call last):
File "train.py", line 342, in
env["grads_variance"] = compute_grads_variance(encodenum, fut, net.decode,op_mask)
File "train.py", line 85, in compute_grads_variance
print(weights.grad_batch.shape)
AttributeError: 'Parameter' object has no attribute 'grad_batch'
```

By the way,I would like to ask if there is a definition of the grad_batch in backpack。

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.