ContinualAI / ContinualAI/avalanche

Unreasonable default for validation stream creation

Open
#1,435 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.1k
Forks
321
PR merge metrics
No merged PRs in 30d

Description

```{python}
benchmark = SplitCifar100(20, shuffle=True)
benchmark = benchmark_with_validation(benchmark, val_size=0.05)
print(benchmark.valid_stream[0].classes_in_this_experience) # Shows 125 samples of the same class
```

The default value to create the validation stream is shuffle=False. This could be reasonable choice if the benchmark dataset is already shuffled (no need to shuffle again in that case). However, when creating it that way, it seems like it's not shuffled.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.