pytest-dev / pytest-dev/pytest-random-order
Suggestion: Add a warning to the README that default random seed is now fixed
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 79
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
After I installed this plugin I began seeing bizarre issues in my tests, like API returning 409 conflict claiming duplicate object creations. It took me a second and some time digging into the logs to figure out that my previous randomness that was in-charge of generating random object identifiers, is no longer as random as I though.
I then connected the dots and figured out that it was the doing of this plugin. I tried changing the scope of the bucket to the minimal level, class, but while the number of these issues dramatically decreased, they still occurred, as I have tests within the same class that require separate random seeds.
I am by no means "blaming" you for this as this plugin worked as it is designed and it is I who didn't fully read and/or understood the implications of using it. I am grateful for its creation and I'm sure many people get high value from it. I do have some suggestions in order to avoid lazy people like me in the future:
- Add a noticeable warning that this plugin manipulates the random seed and that it may disturb tests that rely on the random module.
- If I am not mistaken, the default random bucket scope is
global. Perhaps it would be wise to either set it to the minimum scope by default or even not have a default at all, forcing the user to choose a scope so he'll have some understanding at least on the implications.
Again, I am very grateful for this plugin and I hope my suggestion are taken in the spirit they are written.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the README and any configuration documentation covering random seeds and bucket scope, especially the default global scope. Add a noticeable warning explaining that the plugin changes Python randomness and may affect tests relying on random identifiers. Done means the warning clearly describes this behavior and its possible conflicts; the proposed default-scope change is a separate design discussion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, testing
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100