flaky ci: spurious TimeoutError on macOS runners
- Dominant language
- Rust
- Stars
- 929
- Forks
- 143
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
Here is another timeout error from CI:
```
=================================== FAILURES ===================================
_______________ test_see_new_verified_member_after_going_online ________________
[gw0] darwin -- Python 3.14.6 /Users/runner/work/core/core/python/.tox/py/bin/python
acfactory =
tmp_path = PosixPath('/private/var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/pytest-of-runner/pytest-0/popen-gw0/test_see_new_verified_member_a0')
lp = .Printer object at 0x109be7230>
def test_see_new_verified_member_after_going_online(acfactory, tmp_path, lp):
"""The test for the bug #3836:
- Alice has two devices, the second is offline.
- Alice creates a verified group and sends a QR invitation to Bob.
- Bob joins the group and sends a message there. Alice sees it.
- Alice's second devices goes online, but doesn't see Bob in the group.
"""
ac1, ac2 = acfactory.get_online_accounts(2)
ac2_addr = ac2.get_config("addr")
acfactory.remove_preconfigured_keys()
ac1_offl = acfactory.new_online_configuring_account(cloned_from=ac1)
for ac in [ac1, ac1_offl]:
ac.set_config("bcc_self", "1")
> acfactory.bring_accounts_online()
tests/test_0_complex_or_slow.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:535: in bring_accounts_online
self._acsetup.bring_online()
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:337: in bring_online
self._onconfigure_start_io(acc)
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:351: in _onconfigure_start_io
self.init_imap(acc)
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:370: in init_imap
acc.direct_imap = DirectImap(acc)
^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/deltachat/direct_imap.py:35: in __init__
self.connect()
.tox/py/lib/python3.14/site-packages/deltachat/direct_imap.py:45: in connect
self.conn = MailBox(host, port, ssl_context=ssl.create_default_context())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:401: in __init__
super().__init__()
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:72: in __init__
self.client = self._get_mailbox_client()
^^^^^^^^^^^^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:404: in _get_mailbox_client
return _create_imap4_ssl_client(
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:61: in _create_imap4_ssl_client
return imaplib.IMAP4_SSL(host, port, ssl_context=ssl_context, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1641: in __init__
IMAP4.__init__(self, host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:208: in __init__
self.open(host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1654: in open
IMAP4.open(self, host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:318: in open
self.sock = self._create_socket(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1644: in _create_socket
sock = IMAP4._create_socket(self, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:308: in _create_socket
return socket.create_connection(address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socket.py:874: in create_connection
raise exceptions[0]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('ci-chatmail.testrun.org', 993)
timeout = , source_address = None
all_errors = False
def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
source_address=None, *, all_errors=False):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
A host of '' or port 0 tells the OS to use the default. When a connection
cannot be created, raises the last error if *all_errors* is False,
and an ExceptionGroup of all errors if *all_errors* is True.
"""
host, port = address
exceptions = []
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket(af, socktype, proto)
if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 60] Operation timed out
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socket.py:859: TimeoutError
----------------------------- Captured stdout call -----------------------------
```
For me it looks like this frequently happens on macOS and not other runners.
But I always suspect that there is something wrong with macOS runners possibly having less resources than Linux/Windows runners because it likely requires limited Apple hardware: https://github.com/pytest-dev/pytest-timeout/issues/157
Contributor guide
Research direction
Start with tests/test_0_complex_or_slow.py, especially test_see_new_verified_member_after_going_online, and trace acfactory.bring_accounts_online through testplugin.py and direct_imap.py. Reproduce the timeout on macOS runners and compare it with other runners; done means the cause is identified and the flaky CI failure is reliably prevented or appropriately handled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100