python / python/cpython

`test_file` failed on Windows with a longer repo path in `test_urllib2`

Open
#133,448 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OS-windows tests type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

On Windows, when the CPython repository is cloned to a slightly longer path, for example, C:\Users\xxxxx\Source\cpython — this test case will fail:

ERROR: test_file (test.test_urllib2.HandlerTests.test_file)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\xxxxx\Source\cpython\Lib\encodings\idna.py", line 219, in encode
    result.extend(ToASCII(label))
                  ~~~~~~~^^^^^^^
  File "C:\Users\xxxxx\Source\cpython\Lib\encodings\idna.py", line 115, in ToASCII
    raise UnicodeEncodeError("idna", label, 0, len(label), "label too long")
UnicodeEncodeError: 'idna' codec can't encode characters in position 0-64: label too long

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\xxxxx\Source\cpython\Lib\test\test_urllib2.py", line 872, in test_file
    self.assertRaises(urllib.error.URLError,
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
                      h.file_open, Request(url))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxxxx\Source\cpython\Lib\unittest\case.py", line 813, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxxxx\Source\cpython\Lib\unittest\case.py", line 247, in handle
    callable_obj(*args, **kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "C:\Users\xxxxx\Source\cpython\Lib\urllib\request.py", line 1469, in open_local_file
    localfile = url2pathname(req.full_url, require_scheme=True)
  File "C:\Users\xxxxx\Source\cpython\Lib\urllib\request.py", line 1658, in url2pathname
    if not _is_local_authority(authority):
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "C:\Users\xxxxx\Source\cpython\Lib\urllib\request.py", line 1498, in _is_local_authority
    address = socket.gethostbyname(authority)
  File "C:\Users\xxxxx\Source\cpython\Lib\encodings\idna.py", line 222, in encode
    raise UnicodeEncodeError(
    ...<5 lines>...
    )
UnicodeEncodeError: 'idna' codec can't encode characters in position 8-72: label too long
encoding with 'idna' codec failed

See:
https://github.com/python/cpython/blob/c336f1c3126203fd6c38050df94e9fe8c0d7f2e2/Lib/test/test_urllib2.py#L857-L860

These two URLs will become:

  • file://127.0.0.1:80C:\Users\xxxxx\Source\cpython\build\test_python_worker_17208æ/@test_17208_tmpæ
  • file://somerandomhost.ontheinternet.comC:\Users\xxxxx\Source\cpython\build\test_python_worker_4676æ/@test_4676_tmpæ

Here, 127.0.0.1:80C:\Users\xxxxx\Source\cpython\build\test_python_worker_17208æ is treated as the domain and passed to socket.gethostbyname. However, the part 1:80C:\Users\xxxxx\Source\cpython\build\test_python_worker_17208æ is too long to be a valid domain label. As a result, the idna codec raises an exception, indicating the label is too long.

A patch is on the way.

Linked PRs
  • gh-133449

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Lib/test/test_urllib2.py around test_file, especially the URL setup near lines 857-860 and the assertion near line 872. Run test.test_urllib2.HandlerTests.test_file on Windows with a longer checkout path and compare the resulting URL and exception. Done means the test no longer fails in that environment while retaining its intended URLError assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.