bytedance / bytedance/FullStackBench

Confusion about evaluation datasets

未关闭
#3 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
121
派生
10
PR 合并指标
30 天内没有已合并 PR

描述

When I was looking at the evaluation questions, I found that some PHP questions required the code to be converted into PHP, but the reference code provided was already PHP. Is this intentionally designed to test the model copying ability or is it a design error?

Two examples are as follows:
```
"content": "Please convert the following code to PHP, with the function signature `function order_by_points($nums)`\nNote:\n1. Only implement the target function and import necessary packages, do not generate other code\n\n```php\n\nfunction order_by_points($nums) {\n \n function digits_sum($n) {\n $neg = 1;\n if ($n < 0) {\n $n = -1 * $n;\n $neg = -1;\n }\n $n = array_map('intval', str_split($n));\n $n[0] = $n[0] * $neg;\n return array_sum($n);\n }\n usort($nums, function($a, $b) {\n return digits_sum($a) <=> digits_sum($b);\n });\n return $nums;\n}\n\n```",
"id": 297,

"content": "Please convert the following code to PHP, with the function signature `function do_algebra($operator, $operand)`. Note: Only implement the target function and import necessary packages, do not generate other code.\n\n```php\n\nfunction do_algebra($operator, $operand) {\n\n $expression = strval($operand[0]);\n foreach (array_combine($operator, array_slice($operand, 1)) as $oprt => $oprn) {\n $expression .= $oprt . strval($oprn);\n }\n return eval('return '.$expression.';');\n}\n\n```",
"id": 298,
```

贡献指南

这个仓库没有索引到贡献指南

调研方向

检查评估数据集中的条目297和298,这两个条目就是 issue 中提到的两个示例,并将它们的 PHP 转换提示与嵌入的参考代码进行比较。完成的标准是确定提示与参考代码之间的不匹配是否是有意的,或者相应地更正受影响的数据集条目。

由索引模型根据 Issue 内容生成。

评估

技术栈
php
领域
testing-qa
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。