pytest-dev / pytest-dev/pytest-random-order

Suggestion: Add a warning to the README that default random seed is now fixed

Open
#38 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
79
Forks
14
PR merge metrics
No merged PRs in 30d

Description

After I installed this plugin I began seeing bizarre issues in my tests, like API returning 409 conflict claiming duplicate object creations. It took me a second and some time digging into the logs to figure out that my previous randomness that was in-charge of generating random object identifiers, is no longer as random as I though.

I then connected the dots and figured out that it was the doing of this plugin. I tried changing the scope of the bucket to the minimal level, class, but while the number of these issues dramatically decreased, they still occurred, as I have tests within the same class that require separate random seeds.

I am by no means "blaming" you for this as this plugin worked as it is designed and it is I who didn't fully read and/or understood the implications of using it. I am grateful for its creation and I'm sure many people get high value from it. I do have some suggestions in order to avoid lazy people like me in the future:

  1. Add a noticeable warning that this plugin manipulates the random seed and that it may disturb tests that rely on the random module.
  2. If I am not mistaken, the default random bucket scope is global. Perhaps it would be wise to either set it to the minimum scope by default or even not have a default at all, forcing the user to choose a scope so he'll have some understanding at least on the implications.

Again, I am very grateful for this plugin and I hope my suggestion are taken in the spirit they are written.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the README and any configuration documentation covering random seeds and bucket scope, especially the default global scope. Add a noticeable warning explaining that the plugin changes Python randomness and may affect tests relying on random identifiers. Done means the warning clearly describes this behavior and its possible conflicts; the proposed default-scope change is a separate design discussion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.