poetry publish fails when tool.poetry.source is specified throwing "a repository with name was already added"

Description

When releasing some code we have a system which step 1 is to run a unittest. If that passes, the code is published to our internal packages such that we can use them in production.
The code for the unittest/publish is

set -e
pip install poetry==1.8.3 keyring==24.0.0 keyrings.google-artifactregistry-auth==1.1.1

poetry install
poetry run pytest ./tests/unit_tests

poetry config repositories.my_repo https://my-package/packages"

poetry publish --build -r my_repo

The issue is that the publish part fails when I want to include pytorch with CUDA, thus I have to specify where to get the CUDA-torch from using the tool.poetry.source block.
The publish part throws the warning/error

A repository with name torch-gpu-linux was already added.

and exists with status 1, making the entire pipeline failing since the package cannot be published.

It seems like the repo specified in tool.poetry.source is being added twice, thus failing on the second one.

Workarounds

Not in the .toml-file.

The way we work around it later on, is to skip the torch-part completely and then install it manually later on when the package is downloaded (which easily can lead to bugs).

Poetry Installation Method

pip

Operating System

It is run in a docker-container

Poetry Version

1.8.3

Poetry Configuration

cache-dir = "/root/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
repositories.dinero.url = "https://europe-west4-python.pkg.dev/test-dineroci-cm/ai-packages/"
repositories.torch-gpu-linux.url = "https://download.pytorch.org/whl/cu118"
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"

Python Sysconfig

No response

Example pyproject.toml

The (minimalistic) toml-file is


[tool.poetry]
name = "Foo"
version = "0"
description = ""
authors = ["Me"]
readme = "README.md"

[tool.poetry.dependencies]
python = "~3.11"
sentence-transformers = "^2.7.0"
torch = [
    { version = "^2.3.0", markers = "sys_platform == 'darwin'" },   # macOS, no CUDA
    { version = "^2.3.0+cu118", markers = "sys_platform == 'linux'", source="torch-gpu-linux" }  # Linux with CUDA
]

[tool.poetry.dev-dependencies]
pytest = "^7.2.0"

[[tool.poetry.source]]
name = "my_repo"
url = "https://my-package/packages"
priority = "primary"


[[tool.poetry.source]]
name = "PyPI"
priority = "supplemental"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[[tool.poetry.source]]
name = "torch-gpu-linux"
url = "https://download.pytorch.org/whl/cu118"
priority = "explicit"

Poetry Runtime Logs

poetry publish --build -r my_repo
Loading configuration file /root/.config/pypoetry/config.toml
[11:37:55][Step 1/1] Adding repository repo (https://my-package/packages) and setting it as primary
[11:37:55][Step 1/1] Adding repository PyPI (https://pypi.org/simple/) and setting it as supplemental
[11:37:55][Step 1/1] Adding repository torch-gpu-linux (https://download.pytorch.org/whl/cu118) and setting it as explicit
[11:37:55][Step 1/1] Adding repository torch-gpu-linux (https://download.pytorch.org/whl/cu118) and setting it as explicit
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   Stack trace:
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   12  /usr/local/lib/python3.11/site-packages/cleo/application.py:327 in run
[11:37:56][Step 1/1]        325│ 
[11:37:56][Step 1/1]        326│             try:
[11:37:56][Step 1/1]      → 327│                 exit_code = self._run(io)
[11:37:56][Step 1/1]        328│             except BrokenPipeError:
[11:37:56][Step 1/1]        329│                 # If we are piped to another process, it may close early and send a
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   11  /usr/local/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
[11:37:56][Step 1/1]        188│         self._load_plugins(io)
[11:37:56][Step 1/1]        189│ 
[11:37:56][Step 1/1]      → 190│         exit_code: int = super()._run(io)
[11:37:56][Step 1/1]        191│         return exit_code
[11:37:56][Step 1/1]        192│ 
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   10  /usr/local/lib/python3.11/site-packages/cleo/application.py:431 in _run
[11:37:56][Step 1/1]        429│             io.input.interactive(interactive)
[11:37:56][Step 1/1]        430│ 
[11:37:56][Step 1/1]      → 431│         exit_code = self._run_command(command, io)
[11:37:56][Step 1/1]        432│         self._running_command = None
[11:37:56][Step 1/1]        433│ 
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    9  /usr/local/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
[11:37:56][Step 1/1]        471│ 
[11:37:56][Step 1/1]        472│         if error is not None:
[11:37:56][Step 1/1]      → 473│             raise error
[11:37:56][Step 1/1]        474│ 
[11:37:56][Step 1/1]        475│         return terminate_event.exit_code
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    8  /usr/local/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
[11:37:56][Step 1/1]        455│ 
[11:37:56][Step 1/1]        456│             if command_event.command_should_run():
[11:37:56][Step 1/1]      → 457│                 exit_code = command.run(io)
[11:37:56][Step 1/1]        458│             else:
[11:37:56][Step 1/1]        459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    7  /usr/local/lib/python3.11/site-packages/cleo/commands/base_command.py:117 in run
[11:37:56][Step 1/1]        115│         io.input.validate()
[11:37:56][Step 1/1]        116│ 
[11:37:56][Step 1/1]      → 117│         return self.execute(io) or 0
[11:37:56][Step 1/1]        118│ 
[11:37:56][Step 1/1]        119│     def merge_application_definition(self, merge_args: bool = True) -> None:
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    6  /usr/local/lib/python3.11/site-packages/cleo/commands/command.py:61 in execute
[11:37:56][Step 1/1]         59│ 
[11:37:56][Step 1/1]         60│         try:
[11:37:56][Step 1/1]      →  61│             return self.handle()
[11:37:56][Step 1/1]         62│         except KeyboardInterrupt:
[11:37:56][Step 1/1]         63│             return 1
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    5  /usr/local/lib/python3.11/site-packages/poetry/console/commands/publish.py:52 in handle
[11:37:56][Step 1/1]         50│         from poetry.publishing.publisher import Publisher
[11:37:56][Step 1/1]         51│ 
[11:37:56][Step 1/1]      →  52│         publisher = Publisher(self.poetry, self.io)
[11:37:56][Step 1/1]         53│ 
[11:37:56][Step 1/1]         54│         # Building package first, if told
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    4  /usr/local/lib/python3.11/site-packages/poetry/console/commands/command.py:23 in poetry
[11:37:56][Step 1/1]         21│     def poetry(self) -> Poetry:
[11:37:56][Step 1/1]         22│         if self._poetry is None:
[11:37:56][Step 1/1]      →  23│             return self.get_application().poetry
[11:37:56][Step 1/1]         24│ 
[11:37:56][Step 1/1]         25│         return self._poetry
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    3  /usr/local/lib/python3.11/site-packages/poetry/console/application.py:129 in poetry
[11:37:56][Step 1/1]        127│             project_path = self._io.input.option("directory")
[11:37:56][Step 1/1]        128│ 
[11:37:56][Step 1/1]      → 129│         self._poetry = Factory().create_poetry(
[11:37:56][Step 1/1]        130│             cwd=project_path,
[11:37:56][Step 1/1]        131│             io=self._io,
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    2  /usr/local/lib/python3.11/site-packages/poetry/factory.py:95 in create_poetry
[11:37:56][Step 1/1]         93│ 
[11:37:56][Step 1/1]         94│         poetry.set_pool(
[11:37:56][Step 1/1]      →  95│             self.create_pool(
[11:37:56][Step 1/1]         96│                 config,
[11:37:56][Step 1/1]         97│                 poetry.local_config.get("source", []),
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]    1  /usr/local/lib/python3.11/site-packages/poetry/factory.py:172 in create_pool
[11:37:56][Step 1/1]        170│                 io.write_line(message)
[11:37:56][Step 1/1]        171│ 
[11:37:56][Step 1/1]      → 172│             pool.add_repository(repository, priority=priority)
[11:37:56][Step 1/1]        173│             if repository.name.lower() == "pypi":
[11:37:56][Step 1/1]        174│                 explicit_pypi = True
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   ValueError
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   A repository with name torch-gpu-linux was already added.
[11:37:56][Step 1/1] 
[11:37:56][Step 1/1]   at /usr/local/lib/python3.11/site-packages/poetry/repositories/repository_pool.py:132 in add_repository
[11:37:56][Step 1/1]       128│         Adds a repository to the pool.
[11:37:56][Step 1/1]       129│         """
[11:37:56][Step 1/1]       130│         repository_name = repository.name.lower()
[11:37:56][Step 1/1]       131│         if self.has_repository(repository_name):
[11:37:56][Step 1/1]     → 132│             raise ValueError(
[11:37:56][Step 1/1]       133│                 f"A repository with name {repository_name} was already added."
[11:37:56][Step 1/1]       134│             )
[11:37:56][Step 1/1]       135│ 
[11:37:56][Step 1/1]       136│         if default or secondary:
[11:37:56][Step 1/1] Process exited with code 1

Start with poetry/factory.py:create_pool and poetry/repositories/repository_pool.py:add_repository, then reproduce the failure with the shown pyproject.toml and poetry publish --build -r my_repo command. Done means the torch-gpu-linux source is not added twice and publishing completes without the duplicate-repository error.