man-group / man-group/partialtesting

Current state of Partialtesting

Open
#8 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
77
Forks
9
PR merge metrics
No merged PRs in 30d

Description

I've been using partialtesting and wanted to raise a few issues & open some discussion here as it's rather quiet. I think there's some points that people should be aware of and consider before using this tool. It would be interesting to know if anyone is using it (or similar alternatives) on larger projects.

Here are the things I've noticed:

**Non-python files triggering full test run**
This is documented, non-project files result in a full test run. There's no way to ignore certain files, should a config file change trigger a full test run? Probably. Should a README file change trigger a full test run? Probably not. It seems like it would be useful for the user to specify which files should cause this.

**Can't ignore specific tests**
Similar to the above, if I have some tests which aren't part of my regular test suite (perhaps they are integration tests to run against a specific environment) then there's no easy way to just instruct partialtesting to ignore them.

**CI Pipeline integration**
This is always going to be tricky. If two PRs are opened which have some overlap and both pipelines pass, when one is merged into master the coverage file there will change, and the partialtest in the unmerged PR is no longer valid.

**Partial/Full/No Test**
The way to identify whether to run a partial, full or no test is one of the following:
- No file created
- Empty file created
- File with contents created

It would be much clearer and consistent to handle if a file was always created containing a known & consistent format, like JSON to be parsed with jq.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file, test, or entry point is named. Start by reviewing how partialtesting classifies changed files, selects tests, writes the coverage file, and integrates with CI; agree which of the four requested areas is in scope before implementation. Done should include explicit behavior for the chosen area and a consistent result format if the JSON proposal is adopted.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.