ContinualAI / ContinualAI/avalanche

Cannot use checkpoint.WeightCheckpoint in evaluation metrics

Open
#784 3 comments 0 reactions 0 assignees View on GitHub
bug Evaluation Feature - High Priority logging
Dominant language
Python
Stars
2.1k
Forks
321
PR merge metrics
No merged PRs in 30d

Description

🐛 **The Bug**
Was trying to save model weights using checkpoint evaluation metric, but failed with the following error

```python
TypeError Traceback (most recent call last)
in ()
10
11 print('Computing accuracy on the whole test set')
---> 12 results.append(cl_strategy.eval(scenario.test_stream))

6 frames
/usr/lib/python3.7/copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "__reduce_ex__", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "__reduce__", None)

TypeError: can't pickle generator objects
```

🐜 **To Reproduce**
Follow the tutorial [evaluation page](https://avalanche.continualai.org/from-zero-to-hero-tutorial/05_evaluation)
```python
EvaluationPlugin(
accuracy_metrics(minibatch=True, epoch=True, experience=True, stream=True),
loss_metrics(minibatch=True, epoch=True, experience=True, stream=True),
timing_metrics(epoch=True, epoch_running=True),
forgetting_metrics(experience=True, stream=True),
forward_transfer_metrics(experience=True, stream=True),
confusion_matrix_metrics(
num_classes=scenario.n_classes,
save_image=True,
stream=True
),
[checkpoint.WeightCheckpoint()],
loggers=[interactive_logger, text_logger, tb_logger]
)
```
Then train the model normally, the weight failed to be saved

🐝 **Expected behavior**
Guessing that the model weight would be saved somewhere, but unsure how to use it properly

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.