ContinualAI / ContinualAI/continual-learning-baselines

Table notation for reproducibility

Open
#35 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
358
Forks
48
PR merge metrics
No merged PRs in 30d

Description

I propose to switch the notation. Right now we have:
- ✅ Reproduced
- ❌ Custom setup
- `bug` for bugs

IMO, this is very confusing at a first glance. If I see a big red cross I immediately think there is a problem with the strategy. In this case, everything is actually correct, we just changed some hyperparameters or tested a new benchmark.

Instead we could have two separate columns:
- **Reproduced**: ✅ if correct, ❌ if bugged
- **Reference**: link to the paper, or link to avalanche or `custom` tag if not using any paper.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.