deepseek-ai / deepseek-ai/DeepSeek-Math
Model Size Choices in Evaluation
Open
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 592
- PR merge metrics
- No merged PRs in 30d
Description
For evaluation, the choices of model size are fixed to be ['1b', '7b', '13b', '33b', '34b', '70b'].
/evaluation/run_subset_parallel.py:
`parser.add_argument("--model-size", type=str, choices=['1b', '7b', '13b', '33b', '34b', '70b'], default="7b")`
(1) Why fix the choices?
(2) As I want to evaluate 3B model, can I just add '3b' in the list? Will this cause any problem?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.