ome / ome/openmicroscopy

More non-parallel test assumptions

Open
#6,206 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
222
Forks
105
Avg merge
2h 21m
Merged PRs (30d)
1

Description

Following on from https://github.com/ome/openmicroscopy/pull/6156, I tried running the integration tests as background jobs:

./build.py ... python &
./build.py ... java &
...
wait

but https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/287/#showFailuresLink failed with CLI assumptions:


Stacktrace

self = <test.integration.clitest.test_user.TestUser object at 0x7f52e4b75198>
capsys = <_pytest.capture.CaptureFixture object at 0x7f520acbbd68>
sort_key = None, group_format = None

    @pytest.mark.parametrize("sort_key", sort_keys)
    @pytest.mark.parametrize("group_format", [None, "count", "long"])
    def testList(self, capsys, sort_key, group_format):
        self.args += ["list"]
        if sort_key:
            self.args += ["--sort-by-%s" % sort_key]
        if group_format:
            self.args += ["--%s" % group_format]
        self.cli.invoke(self.args, strict=True)
    
        # Read from the stdout
        out, err = capsys.readouterr()
        ids = get_user_ids(out, sort_key=sort_key)
    
        # Check all users are listed
        if sort_key == 'login':
            sorted_list = sorted(self.users, key=lambda x: x.omeName.val)
        elif sort_key == 'first-name':
            sorted_list = sorted(self.users, key=lambda x: x.firstName.val)
        elif sort_key == 'last-name':
            sorted_list = sorted(self.users, key=lambda x: x.lastName.val)
        elif sort_key == 'email':
            sorted_list = sorted(self.users, key=lambda x: (
                x.email and x.email.val or ""))
        else:
            sorted_list = sorted(self.users, key=lambda x: x.id.val)
>       assert ids == [user.id.val for user in sorted_list]
E       assert [0, 1, 2, 3, 4, 5, ...] == [0, 1, 2, 3, 4, 5, ...]
E         Left contains 2 more items, first extra item: 3127
E         Full diff:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with test.integration.clitest.test_user.TestUser.testList and the failing OMERO-test-integration Jenkins run. Reproduce the CLI test failure with the integration jobs running in parallel, then verify that the user-list assertions pass without including results from another test run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.