ContinualAI / ContinualAI/avalanche
Unreasonable default for validation stream creation
Open
bug
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 321
- PR merge metrics
- No merged PRs in 30d
Description
```{python}
benchmark = SplitCifar100(20, shuffle=True)
benchmark = benchmark_with_validation(benchmark, val_size=0.05)
print(benchmark.valid_stream[0].classes_in_this_experience) # Shows 125 samples of the same class
```
The default value to create the validation stream is shuffle=False. This could be reasonable choice if the benchmark dataset is already shuffled (no need to shuffle again in that case). However, when creating it that way, it seems like it's not shuffled.
Contributor guide
Assessment
This issue has not been assessed yet.