python / python/cpython

multiprocessing.connection.Client deadlocks when trying to connect to a listener without a password

Open
#123,736 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib topic-multiprocessing type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

We set up a multiprocessing.connection.Listener with no authkey. When we try to connect to it with a Client with an authkey, the Client deadlocks. Here is a working example:

import socket
import threading
import time
from multiprocessing.connection import Client, Listener


def _test(*, listener: str | None, client: str | None) -> tuple[Exception | None, Exception | None, list]:
    _autkey_listener = listener.encode() if listener else None
    _authkey_client = client.encode() if client else None

    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.bind(("localhost", 0))
        address = s.getsockname()

    thread_exc = None
    thread_got = []
    client_exc = None

    def target() -> None:
        try:
            with Listener(address, authkey=_autkey_listener) as l:
                c = l.accept()
                while True:
                    value = c.recv()
                    if not value:
                        break
                    thread_got.append(value)
        except Exception as e:
            nonlocal thread_exc
            thread_exc = e

    t = threading.Thread(target=target)
    t.start()

    try:
        with Client(address, authkey=_authkey_client) as c:
            c.send("hello")
            time.sleep(0.1)
            c.send(None)

        t.join()
    except Exception as e:
        client_exc = e

    return thread_exc, client_exc, thread_got


if __name__ == "__main__":
    print(" no passwords ".center(40, "-"))
    print(_test(listener=None, client=None))

    print(" two matching passwords ".center(40, "-"))
    print(_test(listener="password", client="password"))

    print(" client has wrong password ".center(40, "-"))
    print(_test(listener="password", client="wrong"))

    print(" client has no password ".center(40, "-"))
    print(_test(listener="password", client=None))

    # This is the case which deadlocks
    print(" listener has no password ".center(40, "-"))
    print(_test(listener=None, client="password"))

which outputs:

------------- no passwords -------------
(None, None, ['hello'])
-------- two matching passwords --------
(None, None, ['hello'])
------ client has wrong password -------
(AuthenticationError('digest received was wrong'), AuthenticationError('digest sent was rejected'), [])
-------- client has no password --------
(AuthenticationError("expected 'md5' of length 16 got 20"), None, [])
------- listener has no password -------

and then stops.

Upon some investigation, the following bit of code from Client is to blame:

  if authkey is not None:
      answer_challenge(c, authkey)
      deliver_challenge(c, authkey)

since the listener is not configured to deliver the challenge to the connections. I think this would be resolved with an optional timeout kwarg in answer_challenge. If this suggestion is accepted I would be very happy to work on it 👍

CPython versions tested on:

3.9, 3.10, 3.11, 3.12

Operating systems tested on:

macOS, Windows

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with multiprocessing.connection.Client and the answer_challenge path identified in the report, then run the supplied reproducer across its authentication combinations. Done means the listener-without-password and client-with-password case no longer hangs, while the other reported cases retain their demonstrated outcomes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.