python-poetry / python-poetry/poetry

poetry build: crash on encountering a symlink

Open
#3,589 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • OS version and name: macOS High Sierra 10.13.6

  • Poetry version: 1.1.4

Issue

I'm converting my project from a setuptools to Poetry. For building the sdist, Poetry already makes things easier since it excludes files listed in .gitignore files, where this information needs to be duplicated in the MANIFEST.in file when using setuptools.

However, I wanted to go one step further and just include all of my repository's files instead of listing them all, counting on the .gitignore files to exclude unwanted files. To this effect, I am using this configuration in pyproject.toml:

include = [{ path = "**/*", format = "sdist" }]

This should work well, since I typically want all files under source control to be included in the sdist, and this will not require adjustments to pyproject.toml when new files are added to the repository. I might want to exclude certain files such as CI service configuration files, however.

Unfortuntely, Poetry crashes when encountering a symlink in a .tox virtualenv directory with when building the sdist:

  ValueError

  '/Users/brechtm/.pyenv/versions/3.7.7/bin/python3.7' does not start with '/Users/brechtm/Documents/Code/rinohtype'

Since the parent .tox directory is excluded by .gitignore, Poetry shouldn't bother checking any of the files under it.

Full output of: poetry build -vvv -f sdist 
Using virtualenv: /Users/brechtm/Documents/Code/rinohtype/.venv
Building rinohtype (0.5.0-dev.0)
  - Building sdist
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/__init__.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/__main__.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/annotation.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/attribute.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/color.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/csl_formatter.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/dimension.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/document.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/draw.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/element.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/flowable.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/glossary.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/highlight.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/hyphenator.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/image.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/index.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/inline.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/layout.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/number.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/paper.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/paragraph.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/reference.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/resource.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/strings.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/structure.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/style.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/styleds.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/styles.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/table.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/template.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/text.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/util.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/src/rinoh/warnings.py
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.DS_Store
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.bumpversion.cfg
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.coverage
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.coveragerc
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.envrc
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.gitattributes
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.gitignore
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.gitmodules
  - Adding: /Users/brechtm/Documents/Code/rinohtype/.python-version

  Stack trace:

  10  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

   9  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

   8  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│
       173│     def __repr__(self):  # type: () -> str

   7  ~/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│
     →  92│         return self.handle()
        93│
        94│     def handle(self):  # type: () -> Optional[int]

   6  ~/.poetry/lib/poetry/console/commands/build.py:36 in handle
       34│
       35│         builder = Builder(self.poetry)
     → 36│         builder.build(fmt, executable=self.env.python)
       37│

   5  ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/masonry/builder.py:30 in build
       28│
       29│         for builder in builders:
     → 30│             builder(self._poetry, executable=executable).build()
       31│

   4  ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/masonry/builders/sdist.py:77 in build
        75│             tar_dir = "{}-{}".format(self._package.pretty_name, self._meta.version)
        76│
     →  77│             files_to_add = self.find_files_to_add(exclude_build=False)
        78│
        79│             for file in sorted(files_to_add, key=lambda x: x.relative_to_source_root()):

   3  ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/masonry/builders/sdist.py:307 in find_files_to_add
       305│         self, exclude_build=False
       306│     ):  # type: (bool) -> Set[BuildIncludeFile]
     → 307│         to_add = super(SdistBuilder, self).find_files_to_add(exclude_build)
       308│
       309│         # add any additional files, starting with all LICENSE files

   2  ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/masonry/builders/builder.py:166 in find_files_to_add
       164│
       165│                             if not current_file.is_dir() and not self.is_excluded(
     → 166│                                 include_file.relative_to_source_root()
       167│                             ):
       168│                                 to_add.add(include_file)

   1  ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/masonry/builders/builder.py:362 in relative_to_source_root
       360│     def relative_to_source_root(self):  # type(): -> Path
       361│         if self.source_root is not None:
     → 362│             return self.path.relative_to(self.source_root)
       363│         return self.path
       364│

  ValueError

  '/Users/brechtm/.pyenv/versions/3.7.7/bin/python3.7' does not start with '/Users/brechtm/Documents/Code/rinohtype'

  at ~/.pyenv/versions/3.8.3/lib/python3.8/pathlib.py:904 in relative_to
       900│         n = len(to_abs_parts)
       901│         cf = self._flavour.casefold_parts
       902│         if (root or drv) if n == 0 else cf(abs_parts[:n]) != cf(to_abs_parts):
       903│             formatted = self._format_parsed_parts(to_drv, to_root, to_parts)
    →  904│             raise ValueError("{!r} does not start with {!r}"
       905│                              .format(str(self), str(formatted)))
       906│         return self._from_parsed_parts('', root if n == 1 else '',
       907│                                        abs_parts[n:])
       908│

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in poetry/_vendor/py3.8/poetry/core/masonry/builders/builder.py at find_files_to_add and relative_to_source_root, then review the sdist path in builders/sdist.py. Reproduce with poetry build -vvv -f sdist using the shown include setting and a symlink under an excluded .tox directory; done means the build no longer crashes and excluded files remain excluded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.