f-dangel / f-dangel/backpack

Container modules with advanced control flow & modules with multiple inputs

Open
#306 23 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
617
Forks
57
PR merge metrics
No merged PRs in 30d

Description

I have a somewhat complicated `torch.nn.Module`, let's say for arguments sake its structure is a bit like this:

```python
import torch

CustomModule(torch.nn.Module):

def __init__(self):
self.layer1 = OtherCustomModule()
self.layer2 = AnotherCustomModule()
self.layer3 = OtherCustomModule()

def forward(self, inputs)
out = self.layer1(inputs)
out = self.layer2(out)
out = self.layer3(out)
return out
```

Whilst `OtherCustomModule` and `AnotherCustomModule` are themselves composed of some custom functionality, there's some standard layers within them like `nn.Linear`, but there's other stuff going on too.

I've read that as long as the direct children are standard torch modules like `nn.Linear` that `backpack` can detect that and deal with that, however that isn't the case here.

Looking at the example custom module docs with `ScaleModuleBatchGrad`, I'm not sure how i can implement my own class here since `self.layer1` etc are `nn.Module`s not `nn.Parameter`s?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.