exercism / exercism/python-test-runner

When Using subtest, all subtest captured output is written into the Parent Output Property

未關閉
#67 0 則留言 0 個 reaction 已指派 1 人 已被 @BethanyG 認領 在 GitHub 檢視
bug
主要語言
Python
星號
13
分支
20
PR 合併指標
30 天內沒有已合併 PR

描述

If a user has used `print()` or other debugging methods in their code and the tests for that code are using `unittest.subtest`, the captured output for the subtests is being written into the parent `output` property in the `results.json` file -- one set of of output for each subtest. Not an ideal situation.

Steps to reproduce:

Using the `tests/example-partial-failure-with-subtests/` test:
1. Add `print("User output is captured!")` on **line 8** of `example_partial_failure_with_subtests.py`
2. Run ./bin/run.sh example-partial-failure-with-subtests
3. Note that `results.json` has an entry that looks like this for `ExampleSuccessTest.test_abc`:
```JSON
"name": "ExampleSuccessTest.test_abc",
"status": "pass",
"test_code": "input_data = ['frog', 'fish', 'coconut', 'pineapple', 'carrot', 'cucumber', 'grass', 'tree']\nresult_data = [(\"Hello, World!\", param) for param in input_data]\nnumber_of_variants = range(1, len(input_data) + 1)\n\nfor variant, param, result in zip(number_of_variants, input_data, result_data):\n with self.subTest(f\"variation #{variant}\", param=param, result=result):\n self.assertEqual(hello(param), result,",
"task_id": 1,
"output": "User output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!"
}
```

Ideally, we'd write captured output for a subtest into the `output` property for that subtest. In cases where there is not a subtest report (_pytest only reports failed subtests_), we would annotate the parent test `output` to indicate it came from a successful subtest.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。