apache / apache/lucene

Improve random seed portability for test cases [LUCENE-2853]

Open
#3,927 2 comments 0 reactions 0 assignees View on GitHub
affects-version:4.0-ALPHA legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Major tool:build type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Currently, tests get a random seed, and several possibilities can be randomly assigned, or fixed to
specific values with -Dtests.codec or -Dtests.directory.

The default values for these sort of things is 'random'.

I think it would be a great improvement if we:
- improved our code here so that we always consume the same amount from the Random. this means for example, if you specific -Dtests.directory=RAMDirectory, we should consume a wasted int, where we would normally pick one from random.
- in 3.x, i think its worth it to actually consume wasted ints where 4.0 picks the codec.

With these changes, you could ideally (in many situations situations)
1. take a failure and modify parameters to see if its specific to a particular codec or directory, or a general problem
1. take a failure from trunk and see if it affects 3.x

---
Migrated from [LUCENE-2853](https://issues.apache.org/jira/browse/LUCENE-2853) by Robert Muir (@rmuir), updated May 09 2016

Contributor guide

Open the contributing guide

Research direction

Locate the test random-seed handling and the code paths for -Dtests.codec and -Dtests.directory; compare how 3.x and 4.0 select these values. Done means fixed and random selections consume the same amount from Random, allowing failures to be compared across parameter choices and versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.