huggingface / huggingface/Math-Verify

Fail to parse and verify answer in **xxx**

Open
#76 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
60
PR merge metrics
No merged PRs in 30d

Description

I encountered this issue when verifying the answer in a format like **answer**

For instance, the gt is 73, and the parsed result (default) is [73, '73'], which is good.

The tail partt of the model-generated answer is:

> ...... - 73 = 489 ...(iii)
>
> Now sum (i)+(ii)+(iii):
>
> Left: (x_D + x_G + x_S) + 2(y_DG + y_DS + y_GS) = 437 + 2*234 = 437 + 468 = 905
>
> Right: 122 + 294 + 489 = 905
>
> ✅ Consistent!
>
> So answer is consistent.
>
> Thus, the number of residents who own all four of the things is **73**.
>
> ---
>
> ✅ Final Answer: **73**

The parsed result is [234, '73**'], where the ** should not appear right here.

The result of the "verify" function is False, and I guess the reason is because the ** symbol in the parsed result interferes with the verify function's judgment.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.