pypa / pypa/setuptools

UnicodeEncodeError in archive_util

Open
#710 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug help wanted Needs Investigation
Dominant language
Python
Stars
2.9k
Forks
1.4k
Avg merge
1d 1h
Merged PRs (30d)
1

Description

After fixing #709, a new error is revealed in the LC_ALL=C tests in CI.

______________________________ test_unicode_files ______________________________

tarfile_with_unicode = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/unicode-pkg-1.0.tar.gz'

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_unicode_files0')

    def test_unicode_files(tarfile_with_unicode, tmpdir):

        target = tmpdir / 'out'

>       archive_util.unpack_archive(tarfile_with_unicode, six.text_type(target))

setuptools/tests/test_archive_util.py:55: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

setuptools/archive_util.py:53: in unpack_archive

    driver(filename, extract_dir, progress_filter)

setuptools/archive_util.py:169: in unpack_tarfile

    tarobj._extract_member(member, final_dst)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2108: in _extract_member

    self.makefile(tarinfo, targetpath)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tarfile.TarFile object at 0x7f86ff6c9a20>

tarinfo = <TarInfo './._testim\xe4ge.png' at 0x7f86ffefd8e0>

targetpath = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/out/./._testim\xe4ge.png'

    def makefile(self, tarinfo, targetpath):

        """Make a file called targetpath.

            """

        source = self.fileobj

        source.seek(tarinfo.offset_data)

>       with bltn_open(targetpath, "wb") as target:

E       UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 65: ordinal not in range(128)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2148: UnicodeEncodeError

The issue lies in the "Ugh" code in archive_util.py.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with setuptools/archive_util.py around unpack_tarfile and the failing test in setuptools/tests/test_archive_util.py::test_unicode_files. Run that test with LC_ALL=C to reproduce the UnicodeEncodeError, then verify that archive extraction handles the Unicode filename without the failure and that the regression test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.