camelot-dev / camelot-dev/camelot

Issues with tests

Open
#528 0 comments 0 reactions 0 assignees View on GitHub
testing
Dominant language
Python
Stars
3.8k
Forks
546
Avg merge
3d 17h
Merged PRs (30d)
3

Description

The current test coverage configurations could be improved for more meaningful insights.

Integration tests, such as `test_cli`, often inflate coverage metrics by traversing code without thoroughly testing individual components. While integration tests are valuable, unit tests are essential for independently verifying the behaviour of individual code units. For example:, the utils.py has just one unit test!

To ensure that the test coverage is meaningful, the test-measurement should primarily be driven by unit tests, which allow precise control over test scenarios and edge cases. Furthermore, the test-files itself must be excluded from the coverage measurement.

I propose separating unit and integration tests, using markers and/or directories for clear differentiation. While unit tests should be the primary source for coverage metrics, integration tests can still provide supplementary insights into overall code traversal. This balanced approach ensures both high-quality tests and reliable coverage metrics.

I would like to take up this issue.

Ref: #76 #521
Would love to know your comments @vinayak-mehta @bosd

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the current coverage configuration and the tests containing `test_cli`, then inspect `utils.py` and its existing unit test. Determine how unit and integration tests can be distinguished with the proposed markers or directories, and verify that coverage is driven by unit tests while test files are excluded. Done means the coverage report reflects unit-test behavior and integration tests remain available as supplementary checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.