ContinualAI / ContinualAI/avalanche
Cannot create a dataset using the code in notebooks/from-zero-to-hero-tutorial/03_benchmarks.ipynb and the classification dataset cannot switch between train and eval transforms
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 321
- PR merge metrics
- No merged PRs in 30d
Description
These are two issues related to one another.
🐛 **Describe the bug**
Bug 1: `as_classification_dataset` is given 2 arguments in the notebook, but according to the function definition, only one should be provided. `transform_groups` is not a valid input.
Bug 2: even if we give both `train` and `eval` `transform_groups` to an Avalanche dataset, we can't switch between train/eval transform in that dataset.
🐜 **To Reproduce**
Bug 1
```
train_transforms = torchvision.transforms.ToTensor()
eval_transforms = torchvision.transforms.Compose([
torchvision.transforms.ToTensor(),
torchvision.transforms.Resize((32, 32))
])
train_MNIST = as_classification_dataset(
train_MNIST,
transform_groups={
'train': train_transforms,
'eval': eval_transforms
}
)
test_MNIST = as_classification_dataset(
test_MNIST,
transform_groups={
'train': train_transforms,
'eval': eval_transforms
}
)
```
Bug 2
```
# Assume train_MNIST is a classification AvalancheDataset
train_MNIST.train()
print(train_MNIST[0][0].shape)
# torch.Size([1, 28, 28])
train_MNIST.eval()
print(train_MNIST[0][0].shape)
# torch.Size([1, 28, 28]) <-- should be torch.Size([1, 32, 32])
```
🐝 **Expected behavior**
_For Bug 1_
I tried modifying the code in the following ways
```
from avalanche.benchmarks.utils import make_classification_dataset
train_MNIST = make_classification_dataset(
train_MNIST,
transform_groups=transform_groups
)
eval_MNIST = make_classification_dataset(
eval_MNIST,
transform_groups=transform_groups
)
```
and
```
train_MNIST = as_classification_dataset(train_MNIST)
train_MNIST.replace_current_transform_group(transform_groups)
eval_MNIST = as_classification_dataset(eval_MNIST)
eval_MNIST.replace_current_transform_group(transform_groups)
```
and
```
train_MNIST = AvalancheDataset(datasets=train_MNIST, transform_groups=transform_groups)
eval_MNIST = AvalancheDataset(datasets=eval_MNIST, transform_groups=transform_groups)
train_MNIST = as_classification_dataset(train_MNIST)
eval_MNIST = as_classification_dataset(eval_MNIST)
```
1. Approach using `make_classification_dataset` runs, but the functionality of switching between transform groups doesn't work (Bug 2)
2. Approach calling `as_classification_dataset` followed by `.replace_current_transform_group(transform_groups)` doesn't properly wraps the dataset and when enumerating `train_MNIST` the examples are `PIL.Images`.
3. Approach calling `AvalancheDataset` followed by `as_classification_dataset` gives a `DeprecationWarning`
```
DeprecationWarning: AvalancheDataset constructor has been changed. Please check the documentation for the correct usage. You can use `avalanche.benchmarks.utils.make_classification_dataset if you need the old behavior.
```
and similarly to Approach 1, the functionality of switching between transform groups doesn't work (Bug 2).
It seems that Approach 1 may be the correct technique to create a classification AvalancheDataset, but the transformations are not stored in the groups.
_For Bug 2_
If we could switch into `eval` transform group, we expect
```
train_MNIST.eval()
print(train_MNIST[0][0].shape)
# torch.Size([1, 32, 32])
```
🐞 **Screenshots**
Bug 1
🦋 **Additional context**
I'm using `avalanche-lib 0.4.0` and `torch 1.13.1`
Contributor guide
Assessment
This issue has not been assessed yet.