python-poetry / python-poetry/poetry

`poetry add` with `||` operator breaks further poetry commands

Open
#10,569 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/cli area/core kind/bug
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

Description

I am able to add a dependency like this : poetry add 'pytest=^4|^6. However if I run poetry install or similar command that needs to read the pyproject.toml, I see this error:

The requirement is invalid: Unexpected character at column 16

pytest (>=4,<5 || >=6,<7)

Similarly, if I try to add the dependency this this: poetry add 'pytest>=4,<7,!=5.*. I get a similar error when I run something like poetry install:

The requirement is invalid: Unexpected character at column 21

pytest (>=4,<5.dev0 || ==6.*)
Workarounds
  • Manually remove the dependency from pyproject.toml
  • Run poetry update
  • Manually add this: 
    [tool.poetry.dependencies]
pytest = "^4|^6"
  • Run poetry update
Poetry Installation Method

pip inside a virtualenv

Operating System

Ubuntu 24.04

Poetry Version

2.2.1

Poetry Configuration
cache-dir = "/home/rzuckerm/.cache/pypoetry"
data-dir = "/home/rzuckerm/.local/share/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}/python"  # /home/rzuckerm/.local/share/pypoetry/python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/rzuckerm/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Python Sysconfig
sysconfig.log 
cache-dir = "/home/rzuckerm/.cache/pypoetry"
data-dir = "/home/rzuckerm/.local/share/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}/python"  # /home/rzuckerm/.local/share/pypoetry/python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/rzuckerm/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Example pyproject.toml

Before poetry add:

[project]
name = "poetry-example"
version = "0.1.0"
description = ""
authors = [
    {name = "rzuckerm"}
]
readme = "README.md"
requires-python = ">=3.10, <4"
dependencies = []

[tool.poetry]
packages = [{include = "poetry_example", from = "src"}]

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

After poetry add:

[project]
name = "poetry-example"
version = "0.1.0"
description = ""
authors = [
    {name = "rzuckerm"}
]
readme = "README.md"
requires-python = ">=3.10, <4"
dependencies = ["pytest (>=4,<5 || >=6,<7)"]

[tool.poetry]
packages = [{include = "poetry_example", from = "src"}]

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"
Poetry Runtime Logs
poetry-runtime.log 
Stack trace:

1  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/lexer.py:665 in lex
    663│             while True:
    664│                 lexer = self.lexers[parser_state.position]
  → 665│                 yield lexer.next_token(lexer_state, parser_state)
    666│         except EOFError:
    667│             pass

UnexpectedCharacters

No terminal matches '|' in the current parser context, at line 1 col 16

pytest (>=4,<5 || >=6,<7)
               ^
Expected one of:
      * _R_PAREN
      * _MARKER_SEPARATOR
      * _COMMA

Previous tokens: Token('LEGACY_VERSION_CONSTRAINT', '<5')


at ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/lexer.py:598 in next_token
    594│             if not res:
    595│                 allowed = self.scanner.allowed_types - self.ignore_types
    596│                 if not allowed:
    597│                     allowed = {"<END-OF-FILE>"}
  → 598│                 raise UnexpectedCharacters(lex_state.text, line_ctr.char_pos, line_ctr.line, line_ctr.column,
    599│                                            allowed=allowed, token_history=lex_state.last_token and [lex_state.last_token],
    600│                                            state=parser_state, terminals_by_name=self.terminals_by_name)
    601│
    602│             value, type_ = res

The following error occurred when trying to handle this error:


Stack trace:

8  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/version/requirements.py:43 in __init__
     41│
     42│         try:
  →  43│             parsed = _parser.parse(requirement_string)
     44│         except (UnexpectedCharacters, UnexpectedToken) as e:
     45│             raise InvalidRequirementError(

7  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/version/parser.py:31 in parse
     29│             )
     30│
  →  31│         return self._lark.parse(text=text, **kwargs)
     32│

6  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/lark.py:655 in parse
    653│
    654│         """
  → 655│         return self.parser.parse(text, start=start, on_error=on_error)
    656│
    657│

5  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/parser_frontends.py:104 in parse
    102│         kw = {} if on_error is None else {'on_error': on_error}
    103│         stream = self._make_lexer_thread(text)
  → 104│         return self.parser.parse(stream, chosen_start, **kw)
    105│
    106│     def parse_interactive(self, text: Optional[str]=None, start=None):

4  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/parsers/lalr_parser.py:42 in parse
     40│     def parse(self, lexer, start, on_error=None):
     41│         try:
  →  42│             return self.parser.parse(lexer, start)
     43│         except UnexpectedInput as e:
     44│             if on_error is None:

3  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/parsers/lalr_parser.py:88 in parse
     86│         if start_interactive:
     87│             return InteractiveParser(self, parser_state, parser_state.lexer)
  →  88│         return self.parse_from_state(parser_state)
     89│
     90│

2  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/parsers/lalr_parser.py:111 in parse_from_state
    109│             except NameError:
    110│                 pass
  → 111│             raise e
    112│         except Exception as e:
    113│             if self.debug:

1  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/parsers/lalr_parser.py:100 in parse_from_state
     98│         try:
     99│             token = last_token
  → 100│             for token in state.lexer.lex(state):
    101│                 assert token is not None
    102│                 state.feed_token(token)

UnexpectedToken

Unexpected token Token('URI', '||') at line 1, column 16.
Expected one of:
      * _R_PAREN
      * _COMMA
Previous tokens: [Token('LEGACY_VERSION_CONSTRAINT', '<5')]


at ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/_vendor/lark/lexer.py:674 in lex
    670│             # This tests the input against the global context, to provide a nicer error.
    671│             try:
    672│                 last_token = lexer_state.last_token  # Save last_token. Calling root_lexer.next_token will change this to the wrong token
    673│                 token = self.root_lexer.next_token(lexer_state, parser_state)
  → 674│                 raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
    675│             except UnexpectedCharacters:
    676│                 raise e  # Raise the original UnexpectedCharacters. The root lexer raises it with the wrong expected set.
    677│
    678│ ###}

The following error occurred when trying to handle this error:


Stack trace:

16  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/application.py:327 in run
     325│
     326│             try:
   → 327│                 exit_code = self._run(io)
     328│             except BrokenPipeError:
     329│                 # If we are piped to another process, it may close early and send a

15  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:260 in _run
     258│
     259│             try:
   → 260│                 exit_code = super()._run(io)
     261│             except PoetryRuntimeError as e:
     262│                 io.write_error_line("")

14  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/application.py:431 in _run
     429│             io.input.interactive(interactive)
     430│
   → 431│         exit_code = self._run_command(command, io)
     432│         self._running_command = None
     433│

13  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
     471│
     472│         if error is not None:
   → 473│             raise error
     474│
     475│         return terminate_event.exit_code

12  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/application.py:454 in _run_command
     452│
     453│         try:
   → 454│             self._event_dispatcher.dispatch(command_event, COMMAND)
     455│
     456│             if command_event.command_should_run():

11  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/events/event_dispatcher.py:26 in dispatch
      24│
      25│         if listeners:
   →  26│             self._do_dispatch(listeners, event_name, event)
      27│
      28│         return event

10  ~/virtualenvs/poetry/lib/python3.12/site-packages/cleo/events/event_dispatcher.py:85 in _do_dispatch
      83│                 break
      84│
   →  85│             listener(event, event_name, self)
      86│
      87│     def _sort_listeners(self, event_name: str) -> None:

 9  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:578 in configure_env
     576│
     577│         io = event.io
   → 578│         poetry = command.poetry
     579│
     580│         env_manager = EnvManager(poetry, io=io)

 8  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/console/commands/command.py:24 in poetry
      22│     def poetry(self) -> Poetry:
      23│         if self._poetry is None:
   →  24│             return self.get_application().poetry
      25│
      26│         return self._poetry

 7  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:199 in poetry
     197│             return self._poetry
     198│
   → 199│         self._poetry = Factory().create_poetry(
     200│             cwd=self.project_directory,
     201│             io=self._io,

 6  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/factory.py:60 in create_poetry
      58│             io = NullIO()
      59│
   →  60│         base_poetry = super().create_poetry(cwd=cwd, with_groups=with_groups)
      61│
      62│         if version_str := base_poetry.local_config.get("requires-poetry"):

 5  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/factory.py:78 in create_poetry
       76│         assert isinstance(version, str)
       77│         package = self.get_package(name, version)
   →   78│         self.configure_package(
       79│             package, pyproject, poetry_file.parent, with_groups=with_groups
       80│         )

 4  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/factory.py:164 in configure_package
      162│         cls._configure_package_metadata(package, project, tool_poetry, root)
      163│         cls._configure_entry_points(package, project, tool_poetry)
   →  164│         cls._configure_package_dependencies(
      165│             package=package,
      166│             project=project,

 3  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/factory.py:396 in _configure_package_dependencies
      394│             for constraint in dependencies:
      395│                 group.add_dependency(
   →  396│                     Dependency.create_from_pep_508(
      397│                         constraint, relative_to=package.root_dir
      398│                     )

 2  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/packages/dependency.py:370 in create_from_pep_508
     368│                 name += " ;" + rest.split(" ;", 1)[1]
     369│
   → 370│         req = parse_requirement(name)
     371│
     372│         name = req.name

 1  ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/version/requirements.py:118 in parse_requirement
     116│ @functools.cache
     117│ def parse_requirement(requirement_string: str) -> Requirement:
   → 118│     return Requirement(requirement_string)
     119│

InvalidRequirementError

The requirement is invalid: Unexpected character at column 16

pytest (>=4,<5 || >=6,<7)
               ^


at ~/virtualenvs/poetry/lib/python3.12/site-packages/poetry/core/version/requirements.py:45 in __init__
     41│
     42│         try:
     43│             parsed = _parser.parse(requirement_string)
     44│         except (UnexpectedCharacters, UnexpectedToken) as e:
  →  45│             raise InvalidRequirementError(
     46│                 "The requirement is invalid: Unexpected character at column"
     47│                 f" {e.column}\n\n{e.get_context(requirement_string)}"
     48│             )
     49│

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with poetry add 'pytest=^4|^6' and then poetry install; the stack trace points to poetry/core/version/requirements.py and poetry/core/version/parser.py. Trace how the || constraint is parsed, then verify that the generated pyproject.toml can be read by subsequent Poetry commands and that both reported constraint forms work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.