deepseek-ai / deepseek-ai/DeepSeek-Math
regarding minif2f eval
Open
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 592
- PR merge metrics
- No merged PRs in 30d
Description
Hi: from the code
https://github.com/deepseek-ai/DeepSeek-Math/blob/b8b0f8ce093d80bf8e9a641e44142f06d092c305/evaluation/eval/eval_script.py#L171
why it always output True, and if so, how we can make the eval?
thx.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading evaluation/eval/eval_script.py at line 171 and tracing the minif2f evaluation path. Reproduce the reported output and inspect the surrounding result-handling logic; done means identifying why it always reports True and documenting or fixing the evaluation behavior with a verified run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100