Check use of `benchmark_dataset`
Open
- Dominant language
- Python
- Stars
- 5
- Forks
- 3
- Avg merge
- 5d 6h
- Merged PRs (30d)
- 8
Description
We are using this as a key (with "true") to indicate which dataset is the reference.
Is it clear that this means "this one in the benchmark"? Or could this mean "compare this [against the benchmark]"?
Would "is_benchmark" or "is_reference" be any clearer?
Contributor guide
Research direction
Start by locating every use of the `benchmark_dataset` key in the CMEW workflow and determine whether it marks the reference dataset. Review the proposed names with maintainers; done means the meaning is unambiguous and all affected usages consistently use the agreed name.
Written by the indexing model from the issue text.
Assessment
- Domain
- tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100