More non-parallel test assumptions
Open
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 222
- Forks
- 105
- Avg merge
- 2h 21m
- Merged PRs (30d)
- 1
Description
Following on from https://github.com/ome/openmicroscopy/pull/6156, I tried running the integration tests as background jobs:
./build.py ... python &
./build.py ... java &
...
wait
but https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/287/#showFailuresLink failed with CLI assumptions:
Stacktrace
self = <test.integration.clitest.test_user.TestUser object at 0x7f52e4b75198>
capsys = <_pytest.capture.CaptureFixture object at 0x7f520acbbd68>
sort_key = None, group_format = None
@pytest.mark.parametrize("sort_key", sort_keys)
@pytest.mark.parametrize("group_format", [None, "count", "long"])
def testList(self, capsys, sort_key, group_format):
self.args += ["list"]
if sort_key:
self.args += ["--sort-by-%s" % sort_key]
if group_format:
self.args += ["--%s" % group_format]
self.cli.invoke(self.args, strict=True)
# Read from the stdout
out, err = capsys.readouterr()
ids = get_user_ids(out, sort_key=sort_key)
# Check all users are listed
if sort_key == 'login':
sorted_list = sorted(self.users, key=lambda x: x.omeName.val)
elif sort_key == 'first-name':
sorted_list = sorted(self.users, key=lambda x: x.firstName.val)
elif sort_key == 'last-name':
sorted_list = sorted(self.users, key=lambda x: x.lastName.val)
elif sort_key == 'email':
sorted_list = sorted(self.users, key=lambda x: (
x.email and x.email.val or ""))
else:
sorted_list = sorted(self.users, key=lambda x: x.id.val)
> assert ids == [user.id.val for user in sorted_list]
E assert [0, 1, 2, 3, 4, 5, ...] == [0, 1, 2, 3, 4, 5, ...]
E Left contains 2 more items, first extra item: 3127
E Full diff:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test.integration.clitest.test_user.TestUser.testList and the failing OMERO-test-integration Jenkins run. Reproduce the CLI test failure with the integration jobs running in parallel, then verify that the user-list assertions pass without including results from another test run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100