ContinualAI / ContinualAI/avalanche

Add a method to initialize dynamic modules from state_dict

Open
#1,119 4 comments 0 reactions 1 assignee Claimed by @AlbinSou View on GitHub
Feature - Medium Priority good first issue models
Dominant language
Python
Stars
2.1k
Forks
321
PR merge metrics
No merged PRs in 30d

Description

Hi,

**Training**
I initialize `net=MTSimpleCNN()` and train this multi-task model with multi-head classifier on two tasks (one task per experience). After training the model has two heads.
I save the model this model's state dict using standard pytorch `torch.save({'state_dict': net.state_dict()}, filename).` and I want to evaluate it later using a different script.

**Evaluation**
I newly initialize the model `net=MTSimpleCNN()` and load the weights using standard pytorch `net.load_state_dict(torch.load(filename,map_location=device)['state_dict'])`. However the saved model state dict has two heads and the newly initialized one only one head. Keys don't match so I get this error:

`Error(s) in loading state_dict for MTSimpleCNN: Unexpected key(s) in state_dict: "classifier.active_units_T0", "classifier.active_units_T1", "classifier.classifiers.1.active_units", "classifier.classifiers.1.classifier.weight", "classifier.classifiers.1.classifier.bias", "classifier.classifiers.0.active_units"`

is there any easy way to initialize this multi-head network with arbitrary number of heads?

Cheers,
Woj

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.