Tests: check that intended exception messages emerge
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 150
- Forks
- 23
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
To help to avoid bugs whereby an untrue and/or misleading exception message is raised due to some logical issue, such as #732, we should try to test not only that an exception is raised for some cf-python code but that it is the specific one that should be (e.g. ValueError and not just a general Exception) and that the specific error message is correct (bar details that will change which can be selectively ignored), via providing the appropriate string or regex to assertRaisesRegex.
There are (git grep "assertRaises" | wc -l =) 245 uses of assertRaises in the full test suite to test that some exception is raised when it should be, and I imagine most if not all of these can be upgraded in this sense.
See https://github.com/NCAS-CMS/cf-python/pull/732#issuecomment-1977312354 for a small example I have included.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with git grep "assertRaises" to review the 245 existing uses, then inspect the example linked from pull request #732. Identify which assertions can specify the expected exception and message with assertRaisesRegex, and verify the test suite still passes. Done means applicable exception tests check both the intended type and message while allowing documented variable details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100