ASSERT-KTH / ASSERT-KTH/human-eval-java
Incorrect Testing Failures Message
- 主要语言
- Java
- 星标
- 22
- 派生
- 3
- PR 合并指标
- 30 天内没有已合并 PR
描述
I've observed that the test reports for some bugs seem to get the “expected“ value and “but was” value obtained reversed.
For example:
Bug_ID: `ADD`
Buggy_Code:
```java
/* Add two numbers x and y
>>> add(2, 3)
5
>>> add(5, 7)
12 */
public class ADD {
public static int add(int x, int y) {
return x | y;
}
}
```
Test Cases:
```java
public class TEST_ADD {
@org.junit.Test(timeout = 3000)
public void test_0() throws java.lang.Exception {
int result = humaneval.buggy.ADD.add(0, 1);
org.junit.Assert.assertEquals(
result, 1
);
}
@org.junit.Test(timeout = 3000)
public void test_1() throws java.lang.Exception {
int result = humaneval.buggy.ADD.add(1, 0);
org.junit.Assert.assertEquals(
result, 1
);
}
@org.junit.Test(timeout = 3000)
public void test_2() throws java.lang.Exception {
int result = humaneval.buggy.ADD.add(2, 3);
org.junit.Assert.assertEquals(
result, 5
);
}
@org.junit.Test(timeout = 3000)
public void test_3() throws java.lang.Exception {
int result = humaneval.buggy.ADD.add(5, 7);
org.junit.Assert.assertEquals(
result, 12
);
}
@org.junit.Test(timeout = 3000)
public void test_4() throws java.lang.Exception {
int result = humaneval.buggy.ADD.add(7, 5);
org.junit.Assert.assertEquals(
result, 12
);
}
}
```
Current (Incorrect) Testing Failures Message:
```
Tests run: 5, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.032 s <<< FAILURE! - in humaneval.TEST_ADD
test_2(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<3> but was:<5>
at humaneval.TEST_ADD.test_2(TEST_ADD.java:23)
test_3(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<7> but was:<12>
at humaneval.TEST_ADD.test_3(TEST_ADD.java:31)
test_4(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<7> but was:<12>
at humaneval.TEST_ADD.test_4(TEST_ADD.java:39)
```
Correct Testing Failures Message:
```
Tests run: 5, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.032 s <<< FAILURE! - in humaneval.TEST_ADD
test_2(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<5> but was:<3>
at humaneval.TEST_ADD.test_2(TEST_ADD.java:23)
test_3(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<12> but was:<7>
at humaneval.TEST_ADD.test_3(TEST_ADD.java:31)
test_4(humaneval.TEST_ADD) Time elapsed: 0 s <<< FAILURE!
java.lang.AssertionError: expected:<12> but was:<7>
at humaneval.TEST_ADD.test_4(TEST_ADD.java:39)
```
贡献指南
这个仓库没有索引到贡献指南
调研方向
使用所示的 Java 测试用例重现 ADD 示例,并比较当前的失败消息与正确的失败消息。跟踪负责格式化断言失败的测试或报告生成入口点;完成的标准是失败的测试将预期值标记为 expected,并将返回值标记为 actual。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- testing
- Issue 类型
- 缺陷
- 难度
- 2/5
- 预计耗时
- 1-3 小时
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 45/100