camelot-dev / camelot-dev/camelot
Issues with tests
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 546
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 3
Description
The current test coverage configurations could be improved for more meaningful insights.
Integration tests, such as `test_cli`, often inflate coverage metrics by traversing code without thoroughly testing individual components. While integration tests are valuable, unit tests are essential for independently verifying the behaviour of individual code units. For example:, the utils.py has just one unit test!
To ensure that the test coverage is meaningful, the test-measurement should primarily be driven by unit tests, which allow precise control over test scenarios and edge cases. Furthermore, the test-files itself must be excluded from the coverage measurement.
I propose separating unit and integration tests, using markers and/or directories for clear differentiation. While unit tests should be the primary source for coverage metrics, integration tests can still provide supplementary insights into overall code traversal. This balanced approach ensures both high-quality tests and reliable coverage metrics.
I would like to take up this issue.
Ref: #76 #521
Would love to know your comments @vinayak-mehta @bosd
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the current coverage configuration and the tests containing `test_cli`, then inspect `utils.py` and its existing unit test. Determine how unit and integration tests can be distinguished with the proposed markers or directories, and verify that coverage is driven by unit tests while test files are excluded. Done means the coverage report reflects unit-test behavior and integration tests remain available as supplementary checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100