ManimCommunity / ManimCommunity/manim
Logs tests should be refactored/removed
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 40.9k
- Forks
- 3.1k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 25
Description
The title is self-explanatory. Logging tests have been implemented by making a console call to manim, writing the STDOUT to a .txt, and the comparing the generated .txt with the output. This not optimal at all,hard to debug, and kind of ugly.
This should be done instead by calling manim from the test instance, and the logs should be asserted with the caplog fixture of pytest.
Aside, I would be in favor of just nuking those comparison tests, since they take time and don't really make sense as end-to-end tests. They are not enough flexible for changes, which makes them bad tests in my mind.
Instead, we should just test the logging instead of the logs. (i.e, we should test if a DEBUG log has been outputted here, instead of if the value of the log here is equal to ..)
(NB. I'm the one who introduced these tests, so these critics are aimed to me, not at anyone else's work)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the logging tests that invoke manim, capture STDOUT, and compare generated .txt files. Read the pytest caplog fixture documentation, then decide which comparison tests should be removed or converted to logging assertions. Done means the remaining tests check emitted log levels or messages without brittle text-file comparisons.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100