f-dangel / f-dangel/backpack

Extend part of the model

Open
#232 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
617
Forks
57
PR merge metrics
No merged PRs in 30d

Description

Hello,

I am wondering is it possible to extend part of the model, if I only want to get the batch gradient of the last several layers?

I think `model = extend(model)` will waste memory if only the batch gradient of the last several layers is needed.

For example, if I only want to extend the last two layers (let's say the last two layers are fc1 and fc2) of a large model, can I do something like this:

```
model.fc1 = extend(model.fc1)
model.fc2 = extend(model.fc2)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.