python / python/cpython

`test_ssl` is too verbose

Open
#94,609 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tests topic-SSL type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

  • turn loud but ignored server tracebacks into silent exceptions passed straight to a main thread for assertRaises checks. An example of how it's currently (note "ok" in the last line):

    test_connect_ex (test.test_ssl.SimpleBackgroundTests.test_connect_ex) ...  server:  new connection from ('127.0.0.1', 51236)
     
    server:  bad connection attempt from ('127.0.0.1', 51236):
    Traceback (most recent call last):
       File "D:\a\cpython\cpython\Lib\test\test_ssl.py", line 2371, in wrap_conn
        self.sslconn = self.server.context.wrap_socket(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 455, in wrap_socket
        return self.sslsocket_class._create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 1013, in _create
        self.do_handshake()
        ^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 1284, in do_handshake
        self._sslobj.do_handshake()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
    ok
    
  • stop printing a dialog between a client and a server; use the same assertion mechanism instead. An example:

    test_alpn_protocols (test.test_ssl.ThreadedTests.test_alpn_protocols) ...  server:  new connection from ('127.0.0.1', 51305)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51307)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51309)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51311)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
    ok
    

Pitch

While (almost) all tests keep silence, test_ssl reports every nit into a console making the log bloated (>1200 lines) to the degree of being unusable for failure tracking.

In addition, all these reports must be looked through manually. Assertions should be used instead.

Linked PRs
  • gh-92475

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Lib/test/test_ssl.py, especially the server and client interaction paths shown in the issue, and inspect how test.test_ssl currently reports tracebacks and dialogue. Run the relevant test.test_ssl cases to establish the noisy output; done means ignored server failures reach the main thread for assertions and client-server exchanges no longer print progress, while the linked PR should be checked first.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.