exercism / exercism/python-test-runner

When Using subtest, all subtest captured output is written into the Parent Output Property

Aperta
#67 0 commenti 0 reazioni 1 assegnatario Rivendicata da @BethanyG Vedi su GitHub
bug
Lingua principale
Python
Stelle
13
Fork
20
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

If a user has used `print()` or other debugging methods in their code and the tests for that code are using `unittest.subtest`, the captured output for the subtests is being written into the parent `output` property in the `results.json` file -- one set of of output for each subtest. Not an ideal situation.

Steps to reproduce:

Using the `tests/example-partial-failure-with-subtests/` test:
1. Add `print("User output is captured!")` on **line 8** of `example_partial_failure_with_subtests.py`
2. Run ./bin/run.sh example-partial-failure-with-subtests
3. Note that `results.json` has an entry that looks like this for `ExampleSuccessTest.test_abc`:
```JSON
"name": "ExampleSuccessTest.test_abc",
"status": "pass",
"test_code": "input_data = ['frog', 'fish', 'coconut', 'pineapple', 'carrot', 'cucumber', 'grass', 'tree']\nresult_data = [(\"Hello, World!\", param) for param in input_data]\nnumber_of_variants = range(1, len(input_data) + 1)\n\nfor variant, param, result in zip(number_of_variants, input_data, result_data):\n with self.subTest(f\"variation #{variant}\", param=param, result=result):\n self.assertEqual(hello(param), result,",
"task_id": 1,
"output": "User output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!\nUser output is captured!"
}
```

Ideally, we'd write captured output for a subtest into the `output` property for that subtest. In cases where there is not a subtest report (_pytest only reports failed subtests_), we would annotate the parent test `output` to indicate it came from a successful subtest.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.