pcdshub / pcdshub/lightpath

Intermittent test failures from pcds-envs

Open
#169 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4
Forks
9
Avg merge
7d 14h
Merged PRs (30d)
1

Description

Expected Behavior

Tests should always pass!

Current Behavior

See https://github.com/pcdshub/pcds-envs/issues/253

lightpath/tests/test_gui.py::test_upstream_check times out occasionally (10 minutes)

________________________________ test_filtering ________________________________
lightapp = <lightpath.ui.gui.LightApp object at 0x7faf975e80d0>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7faf9c1b37f0>
    def test_filtering(lightapp: LightApp, monkeypatch):
        lightapp.destination_combo.setCurrentIndex(4)  # set current to MEC
        # Create mock functions
        for row in lightapp.rows:
            monkeypatch.setattr(row[0], 'setHidden', Mock())
        # Initialize properly with nothing hidden
        lightapp.filter()
        for row in lightapp.rows:
            row[0].setHidden.assert_called_with(False)
        # Insert at least one device then hide
        device_row = lightapp.rows[2][0]
        device_row.device.insert()
        lightapp.remove_check.setChecked(False)
        # Reset mock
        for row in lightapp.rows:
            row[0].setHidden.reset_mock()
        lightapp.filter()
        for row in lightapp.rows:
            if row[0].device.get_lightpath_state().removed:
>               row[0].setHidden.assert_called_with(True)
lightpath/tests/test_gui.py:90: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <Mock id='140392215288080'>, args = (True,), kwargs = {}
expected = call(True), actual = call(False)
_error_message = <function NonCallableMock.assert_called_with.<locals>._error_message at 0x7faf9762c9d0>
cause = None
    def assert_called_with(self, /, *args, **kwargs):
        """assert that the last call was made with the specified arguments.
    
        Raises an AssertionError if the args and keyword args passed in are
        different to the last call to the mock."""
        if self.call_args is None:
            expected = self._format_mock_call_signature(args, kwargs)
            actual = 'not called.'
            error_message = ('expected call not found.\nExpected: %s\nActual: %s'
                    % (expected, actual))
            raise AssertionError(error_message)
    
        def _error_message():
            msg = self._format_mock_failure_message(args, kwargs)
            return msg
        expected = self._call_matcher(_Call((args, kwargs), two=True))
        actual = self._call_matcher(self.call_args)
        if actual != expected:
            cause = expected if isinstance(expected, Exception) else None
>           raise AssertionError(_error_message()) from cause
E           AssertionError: expected call not found.
E           Expected: mock(True)
E           Actual: mock(False)
../../../miniconda/envs/pcds-next-incr/lib/python3.9/unittest/mock.py:907: AssertionError
________________________________ test_callback _________________________________

path = BeamPath(range=(0.0, 30.0), devices=11)

    def test_callback(path: BeamPath):
        # Create mock callback
        cb = Mock()
        # Subscribe to event changes
        path.subscribe(cb, run=False)
        # Change state of beampath
        path.devices[4].insert()
        # Assert callback has been run
>       assert cb.called
E       AssertionError: assert False
E        +  where False = <Mock id='140651858531136'>.called

lightpath/tests/test_path.py:207: AssertionError

Possible Solution

Figure out how to reproduce these offline or otherwise figure out why they might be happening and mitigate the race condition risk.

Steps to Reproduce (for bugs)

  1. Run the test suite a lot

Context

Probably low priority. This adds some noise to the pcds-envs unit testing suite.

Your Environment

pcds-5.5.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by repeatedly running the affected tests in lightpath/tests/test_gui.py and lightpath/tests/test_path.py, focusing on test_upstream_check, test_filtering, and test_callback. Compare the failures with pcds-envs issue 253 and determine why the intermittent behavior occurs. Done means the tests run reliably without the reported timeout or assertion failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.