jerry-git / jerry-git/pytest-split

Split results in 0 selected tests, which leads to failed GitHub action

Open
#95 8 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
325
Forks
56
PR merge metrics
No merged PRs in 30d

Description

Hi,

We have integrated `pytest-split` as part of our CI pipeline and noticed the following behaviour.

We were running our tests in 7 splits, but after updating the `.test_durations` script (new tests being added), the 7th split did not select any test. This lead to a failed CI run since all the tests were deselected (see below).

![image](https://github.com/jerry-git/pytest-split/assets/38454520/0a7f76f2-e590-4baa-9de0-d51e1d3be752)

Any idea why this could happen?

We tried to create more splits after this error was raised (increased from 7 to 10) and everything worked fine.

**We would like to understand why the 7th split did not select any test when `--splits 7` but everything worked well when `--splits 10`.**

Thanks in advance. Your package has become very useful in speeding up our CICD pipelines.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the split-selection logic that reads `.test_durations` and the `--splits` option; reproduce the reported run with 7 splits and compare it with 10. Check the GitHub Actions output for the zero-selected-test case and add a regression test if the behavior is a bug. Done means the cause and expected handling are documented and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.