square / square/pylink

Tests fail on some architectures (using unittest.mock)

Open
#175 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
430
Forks
151
Avg merge
5d 3h
Merged PRs (30d)
1

Description

I am trying to package for openSUSE this as it is a dependency of pynitrokey.

As openSUSE tries to remove use of python-mock, I simply switched all tests to unittest.mock:

find tests -type f -exec sed -i 's/from mock/from unittest.mock/g' {} +                                                                                                                      
find tests -type f -exec sed -i 's/import mock/import unittest.mock as mock/g' {} +

Worked like charm (and might be usable for #149), as the tests pass successfully on x86_64.

However there are some that fail on non-x86:

armv7l

  • tests/unit/test_library.py::TestLibrary::test_initialize_windows

i586

  • tests/unit/test_library.py::TestLibrary::test_initialize_windows

ppc64

  • tests/unit/unlockers/test_unlock_kinetis.py::TestUnlockKinetis::test_unlock_kinetis_read_fail
  • tests/unit/unlockers/test_unlock_kinetis.py::TestUnlockKinetis::test_unlock_kinetis_success

s390x

  • tests/unit/unlockers/test_unlock_kinetis.py::TestUnlockKinetis::test_unlock_kinetis_read_fail
  • tests/unit/unlockers/test_unlock_kinetis.py::TestUnlockKinetis::test_unlock_kinetis_success

Errors

[   32s] =================================== FAILURES ===================================
[   32s] _____________________ TestLibrary.test_initialize_windows ______________________
[   32s] 
[   32s] self = <tests.unit.test_library.TestLibrary testMethod=test_initialize_windows>
[   32s] mock_ctypes = <MagicMock name='ctypes' id='4116548240'>
[   32s] mock_find_library = <MagicMock name='find_library' id='4116458680'>
[   32s] mock_open = <MagicMock name='open' id='4116527232'>
[   32s] 
[   32s]     @mock.patch('sys.platform', new='windows')
[   32s]     @mock.patch('pylink.library.open')
[   32s]     @mock.patch('os.remove', new=mock.Mock())
[   32s]     @mock.patch('tempfile.NamedTemporaryFile', new=mock.Mock())
[   32s]     @mock.patch('ctypes.util.find_library')
[   32s]     @mock.patch('pylink.library.ctypes')
[   32s]     def test_initialize_windows(self, mock_ctypes, mock_find_library, mock_open):
[   32s]         """Tests creating a library on a Windows machine.
[   32s]     
[   32s]         Args:
[   32s]           self (TestLibrary): the ``TestLibrary`` instance
[   32s]           mock_ctypes (Mock): a mocked version of the ctypes library
[   32s]           mock_find_library (Mock): mock for mocking the
[   32s]             ``ctypes.util.find_library()`` call
[   32s]           mock_open (Mock): mock for mocking the call to ``open()``
[   32s]     
[   32s]         Returns:
[   32s]           ``None``
[   32s]         """
[   32s]         mock_windll = mock.Mock()
[   32s]         mock_windll.__getitem__ = mock.Mock()
[   32s]     
[   32s]         mock_cdll = mock.Mock()
[   32s]         mock_cdll.__getitem__ = mock.Mock()
[   32s]     
[   32s]         mock_ctypes.windll = mock_windll
[   32s]         mock_ctypes.cdll = mock_cdll
[   32s]         mock_find_library.return_value = self.lib_path
[   32s]     
[   32s]         lib = library.Library()
[   32s]         lib.unload = mock.Mock()
[   32s]     
[   32s] >       mock_find_library.assert_called_once_with(library.Library.WINDOWS_64_JLINK_SDK_NAME)
[   32s] 
[   32s] tests/unit/test_library.py:250: 
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] /usr/lib/python3.9/unittest/mock.py:919: in assert_called_once_with
[   32s]     return self.assert_called_with(*args, **kwargs)
[   32s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   32s] 
[   32s] self = <MagicMock name='find_library' id='4116458680'>, args = ('JLink_x64',)
[   32s] kwargs = {}, expected = call('JLink_x64'), actual = call('JLinkARM')
[   32s] _error_message = <function NonCallableMock.assert_called_with.<locals>._error_message at 0xf5f42928>
[   32s] cause = None
[   32s] 
[   32s]     def assert_called_with(self, /, *args, **kwargs):
[   32s]         """assert that the last call was made with the specified arguments.
[   32s]     
[   32s]         Raises an AssertionError if the args and keyword args passed in are
[   32s]         different to the last call to the mock."""
[   32s]         if self.call_args is None:
[   32s]             expected = self._format_mock_call_signature(args, kwargs)
[   32s]             actual = 'not called.'
[   32s]             error_message = ('expected call not found.\nExpected: %s\nActual: %s'
[   32s]                     % (expected, actual))
[   32s]             raise AssertionError(error_message)
[   32s]     
[   32s]         def _error_message():
[   32s]             msg = self._format_mock_failure_message(args, kwargs)
[   32s]             return msg
[   32s]         expected = self._call_matcher(_Call((args, kwargs), two=True))
[   32s]         actual = self._call_matcher(self.call_args)
[   32s]         if actual != expected:
[   32s]             cause = expected if isinstance(expected, Exception) else None
[   32s] >           raise AssertionError(_error_message()) from cause
[   32s] E           AssertionError: expected call not found.
[   32s] E           Expected: find_library('JLink_x64')
[   32s] E           Actual: find_library('JLinkARM')
[   32s] 
[   32s] /usr/lib/python3.9/unittest/mock.py:907: AssertionError
[   25s] =================================== FAILURES ===================================
[   25s] _______________ TestUnlockKinetis.test_unlock_kinetis_read_fail ________________
[   25s] 
[   25s] self = <tests.unit.unlockers.test_unlock_kinetis.TestUnlockKinetis testMethod=test_unlock_kinetis_read_fail>
[   25s] mock_sleep = <MagicMock name='sleep' id='140735395602928'>
[   25s] 
[   25s]     @mock.patch('time.sleep')
[   25s]     def test_unlock_kinetis_read_fail(self, mock_sleep):
[   25s]         """Tests unlock Kinetis failing during a read.
[   25s]     
[   25s]         At any point, the unlock can fail if it tries to read from the an
[   25s]         MDM-AP register and a fault occurs.
[   25s]     
[   25s]         Args:
[   25s]           self (TestUnlockKinetis): the `TestUnlockKinetis` instance
[   25s]           mock_sleep (Mock): mocked `time.sleep()` function
[   25s]     
[   25s]         Returns:
[   25s]           `None`
[   25s]         """
[   25s]         mock_jlink = mock.Mock()
[   25s]         mock_jlink.tif = enums.JLinkInterfaces.SWD
[   25s]     
[   25s]         flags = (0x2 << 28) | (0xBA01 << 12) | 1
[   25s]         mock_jlink.coresight_read.side_effect = [flags, 0xFF]
[   25s]     
[   25s]         mock_jlink.swd_write.return_value = 0   # ACK
[   25s]         mock_jlink.swd_read8.return_value = -1  # Invalid Read
[   25s]         mock_jlink.swd_read32.return_value = 2  # Data
[   25s]     
[   25s]         res = unlock.unlock_kinetis(mock_jlink)
[   25s]         self.assertFalse(res)
[   25s]     
[   25s] >       self.assertEqual(1, mock_jlink.set_reset_pin_low.call_count)
[   25s] E       AssertionError: 1 != 0
[   25s] 
[   25s] tests/unit/unlockers/test_unlock_kinetis.py:157: AssertionError
[   25s] ________________ TestUnlockKinetis.test_unlock_kinetis_success _________________
[   25s] 
[   25s] self = <tests.unit.unlockers.test_unlock_kinetis.TestUnlockKinetis testMethod=test_unlock_kinetis_success>
[   25s] mock_sleep = <MagicMock name='sleep' id='140735394886032'>
[   25s] 
[   25s]     @mock.patch('time.sleep')
[   25s]     def test_unlock_kinetis_success(self, mock_sleep):
[   25s]         """Tests unlock Kinetis succesfully unlocking the device.
[   25s]     
[   25s]         Args:
[   25s]           self (TestUnlockKinetis): the `TestUnlockKinetis` instance
[   25s]           mock_sleep (Mock): mocked `time.sleep()` function
[   25s]     
[   25s]         Returns:
[   25s]           `None`
[   25s]         """
[   25s]         mock_jlink = mock.Mock()
[   25s]         mock_jlink.tif = enums.JLinkInterfaces.SWD
[   25s]     
[   25s]         flags = (0x2 << 28) | (0xBA01 << 12) | 1
[   25s]         mock_jlink.coresight_read.side_effect = [flags, 0xFF]
[   25s]     
[   25s]         ack = swd.Response.STATUS_ACK
[   25s]         wait = swd.Response.STATUS_WAIT
[   25s]     
[   25s]         mocked_data = mock.Mock()
[   25s]         mocked_data.read_data = [
[   25s]             0x0,  # Wait
[   25s]             0x1,  # Ack
[   25s]             0x3,  # Flash Ready Bit Set
[   25s]             0x1,  # Ack
[   25s]             0x3,  # Erase Command Bit Set
[   25s]             0x1,  # Ack
[   25s]             0x0,  # Flash Erase In Progress Bit Cleared
[   25s]         ]
[   25s]     
[   25s]         mocked_data.status_and_parity_data = [
[   25s]             ack,   # Error clearing write request.
[   25s]             ack,   # Selecting the MDM-AP Status Register
[   25s]             wait,  # First poll, wait
[   25s]             ack,   # Second poll, continue
[   25s]             0x1,   # Parity
[   25s]             ack,   # Flash Ready ACK
[   25s]             0x0,   # Flash Ready Parity
[   25s]             0x1,   # Mass Erase Request
[   25s]             ack,   # Erase Command ACK
[   25s]             0x1,   # Erase Command Parity
[   25s]             ack,   # Erase Command ACK
[   25s]             0x0,   # Erase Command Parity
[   25s]             ack,   # Flash Erase Command ACK
[   25s]             0x1,   # Flash Erase Command Parity
[   25s]             ack,   # Flash Erase Command ACK
[   25s]             0x0,   # Flash Erase Command Parity
[   25s]         ]
[   25s]     
[   25s]         # ACK
[   25s]         mock_jlink.swd_write.return_value = 0
[   25s]     
[   25s]         # Status and Parity
[   25s]         mock_jlink.swd_read8.side_effect = mocked_data.status_and_parity_data
[   25s]     
[   25s]         # Data
[   25s]         mock_jlink.swd_read32.side_effect = mocked_data.read_data
[   25s]     
[   25s]         res = unlock.unlock_kinetis(mock_jlink)
[   25s] >       self.assertTrue(res)
[   25s] E       AssertionError: False is not true
[   25s] 
[   25s] tests/unit/unlockers/test_unlock_kinetis.py:221: AssertionError

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the failing tests in tests/unit/test_library.py and tests/unit/unlockers/test_unlock_kinetis.py on the affected architectures. Compare the platform-dependent library selection and Kinetis unlock behavior with the assertions and mocked sequences; done means all listed tests pass on armv7l, i586, ppc64, and s390x.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
embedded-iot, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.