ContinualAI / ContinualAI/avalanche

MASPlugin errors if SGD Loss is zero

Open
#1,676 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.1k
Forks
321
PR merge metrics
No merged PRs in 30d

Description

In the `MASPlugin` in the `before_backward` callback there is a check to see if the loss has been generated from the `SGDUpdate` class's `training_epoch` function.

```
if not strategy.loss:
raise ValueError("Loss is not available")
```

However, at times when dealing with a small amount of data in an experience, if the classifier is near perfect in its logits output, PyTorch may return `tensor(0., device='cuda:0', grad_fn=)` for its loss output from `nn.CrossEntropyLoss()` due to numerical precision. Since a tensor with all zeroes evaluates to False when converted to a boolean, in this case the MAS algorithm errors even though the SGD update has actually occurred correctly.

Here are a few solutions:
- Use a `if not strategy.loss.requires_grad` check instead of a `if not strategy.loss` check
- Replace strategy.loss to be None initially, do not use self._make_empty_loss(), and in the `MASPlugin` check `if strategy.loss is not None`

Contributor guide

Open the contributing guide

Research direction

Start in MASPlugin's before_backward callback and trace strategy.loss back to SGDUpdate.training_epoch; reproduce the small-experience case where nn.CrossEntropyLoss() returns a zero tensor. Validate the proposed loss-availability behavior, ensuring a valid zero loss does not raise while an unavailable loss still does.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.