ContinualAI / ContinualAI/avalanche
new_instances_benchmark problems
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 321
- PR merge metrics
- No merged PRs in 30d
Description
🐛 **Describe the bug**
One of this problem comprises a bug, which gives the following error when calling classes_in_this_experience on an experience. The same bug already happened for class_incremental_benchmark in combination with benchmark_with_validation_stream (here I am also using benchmark_with_validation_stream).
```
AttributeError: 'DatasetExperience' object has no attribute 'classes_in_this_experience'
```
Otherwise, these are just general things about this method make it unusable. Ideally, when calling this, I would like to do something like this:
```
train_dataset = TinyImagenet(
root=dataset_root, train=True, transform=train_transform
)
test_dataset = TinyImagenet(
root=dataset_root, train=False, transform=eval_transform
)
benchmark = new_instances_benchmark(
train_dataset,
test_dataset,
balance_experiences=True,
shuffle=shuffle,
num_experiences=n_experiences,
seed=seed,
)
benchmark = benchmark_with_validation_stream(
benchmark, validation_size=val_size, shuffle=True
)
```
However, if I want to get it work, I currently have to do something like this
```
train_dataset = TinyImagenet(
root=dataset_root, train=True, transform=train_transform
)
test_dataset = TinyImagenet(
root=dataset_root, train=False, transform=eval_transform
)
train_dataset = ClassificationDataset(
train_dataset,
data_attributes=[
DataAttribute(train_dataset.targets, "targets"),
DataAttribute([0] * len(train_dataset), "targets_task_labels", use_in_getitem=True),
],
)
test_dataset = ClassificationDataset(
test_dataset,
data_attributes=[
DataAttribute(test_dataset.targets, "targets"),
DataAttribute([0] * len(test_dataset), "targets_task_labels", use_in_getitem=True),
],
)
benchmark = new_instances_benchmark(
train_dataset,
test_dataset,
balance_experiences=True,
shuffle=shuffle,
num_experiences=n_experiences,
seed=seed,
)
benchmark = benchmark_with_validation_stream(
benchmark, validation_size=val_size, shuffle=True
)
```
Contributor guide
Assessment
This issue has not been assessed yet.