chatmail / chatmail/core

flaky ci: spurious TimeoutError on macOS runners

Open
#8,505 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
929
Forks
143
Avg merge
1d 20h
Merged PRs (30d)
53

Description

Here is another timeout error from CI:
```
=================================== FAILURES ===================================
_______________ test_see_new_verified_member_after_going_online ________________
[gw0] darwin -- Python 3.14.6 /Users/runner/work/core/core/python/.tox/py/bin/python

acfactory =
tmp_path = PosixPath('/private/var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/pytest-of-runner/pytest-0/popen-gw0/test_see_new_verified_member_a0')
lp = .Printer object at 0x109be7230>

def test_see_new_verified_member_after_going_online(acfactory, tmp_path, lp):
"""The test for the bug #3836:
- Alice has two devices, the second is offline.
- Alice creates a verified group and sends a QR invitation to Bob.
- Bob joins the group and sends a message there. Alice sees it.
- Alice's second devices goes online, but doesn't see Bob in the group.
"""
ac1, ac2 = acfactory.get_online_accounts(2)
ac2_addr = ac2.get_config("addr")
acfactory.remove_preconfigured_keys()
ac1_offl = acfactory.new_online_configuring_account(cloned_from=ac1)
for ac in [ac1, ac1_offl]:
ac.set_config("bcc_self", "1")
> acfactory.bring_accounts_online()

tests/test_0_complex_or_slow.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:535: in bring_accounts_online
self._acsetup.bring_online()
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:337: in bring_online
self._onconfigure_start_io(acc)
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:351: in _onconfigure_start_io
self.init_imap(acc)
.tox/py/lib/python3.14/site-packages/deltachat/testplugin.py:370: in init_imap
acc.direct_imap = DirectImap(acc)
^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/deltachat/direct_imap.py:35: in __init__
self.connect()
.tox/py/lib/python3.14/site-packages/deltachat/direct_imap.py:45: in connect
self.conn = MailBox(host, port, ssl_context=ssl.create_default_context())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:401: in __init__
super().__init__()
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:72: in __init__
self.client = self._get_mailbox_client()
^^^^^^^^^^^^^^^^^^^^^^^^^^
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:404: in _get_mailbox_client
return _create_imap4_ssl_client(
.tox/py/lib/python3.14/site-packages/imap_tools/mailbox.py:61: in _create_imap4_ssl_client
return imaplib.IMAP4_SSL(host, port, ssl_context=ssl_context, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1641: in __init__
IMAP4.__init__(self, host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:208: in __init__
self.open(host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1654: in open
IMAP4.open(self, host, port, timeout)
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:318: in open
self.sock = self._create_socket(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:1644: in _create_socket
sock = IMAP4._create_socket(self, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/imaplib.py:308: in _create_socket
return socket.create_connection(address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socket.py:874: in create_connection
raise exceptions[0]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('ci-chatmail.testrun.org', 993)
timeout = , source_address = None
all_errors = False

def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
source_address=None, *, all_errors=False):
"""Connect to *address* and return the socket object.

Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
A host of '' or port 0 tells the OS to use the default. When a connection
cannot be created, raises the last error if *all_errors* is False,
and an ExceptionGroup of all errors if *all_errors* is True.
"""

host, port = address
exceptions = []
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket(af, socktype, proto)
if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 60] Operation timed out

/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/socket.py:859: TimeoutError
----------------------------- Captured stdout call -----------------------------
```

For me it looks like this frequently happens on macOS and not other runners.
But I always suspect that there is something wrong with macOS runners possibly having less resources than Linux/Windows runners because it likely requires limited Apple hardware: https://github.com/pytest-dev/pytest-timeout/issues/157

Contributor guide

Open the contributing guide

Research direction

Start with tests/test_0_complex_or_slow.py, especially test_see_new_verified_member_after_going_online, and trace acfactory.bring_accounts_online through testplugin.py and direct_imap.py. Reproduce the timeout on macOS runners and compare it with other runners; done means the cause is identified and the flaky CI failure is reliably prevented or appropriately handled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.