benchopt / benchopt/benchmark_bci
[EHN] the benchmark results are fluctuating in different executions
Open
- Dominant language
- Python
- Stars
- 19
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
It's not much, and it doesn't affect the rank, but this fluctuation should be eliminated/controlled.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the benchmark repeatedly and comparing the reported results across executions. Inspect the benchmark configuration and execution path for sources of nondeterminism; done means repeated runs produce controlled, consistent results without changing the ranking.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100