jerry-git / jerry-git/pytest-split

feature request: order tests by slowest-first

Open
#89 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
325
Forks
56
PR merge metrics
No merged PRs in 30d

Description

in long running test suites (especially with xdist), it's useful to run the slowest tests first so that you end up with better load. the following code seems to do so:
```
def pytest_collection_modifyitems(config, items):
try:
with open(".test_durations") as f:
s = f.read()
durations = json.loads(s)
except FileNotFoundError:
durations = {}

timings = {}
for item in items:
path, _, reqname = item.location
request_id = f"{path}::{reqname}"
timing = durations.get(request_id, None)
if timing is not None:
timings[item] = timing

meantime = sum(timings.values()) / len(timings)
# sort by highest time first
items.sort(key=lambda item: -timings.get(item, meantime))
```
but i was thinking it might be good for this to be available in the pytest-split plugin itself, so we don't need to add this same function across different projects.

alternatively, i was thinking publishing this as a plugin myself, if pytest-split does not want to add it.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the pytest-split plugin entry points around collection modification and the existing handling of .test_durations. Check how xdist-related splitting is currently implemented and whether the proposed pytest_collection_modifyitems hook fits the plugin's flow. Done means the plugin can order tests slowest-first without requiring repeated project-specific code.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.