Missing credentials when attempting to install dependency from private Gitlab project

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• Ubuntu 20,04 WSL2:
• 1.2.0b2:

[tool.poetry]
name = "<package_name>"
version = "0.0.0"
description = "test"
authors = ["Chris Buffett <email@domain.com>"]
license = "Proprietary"

[tool.poetry.dependencies]
python = "^3.7"
requests = "^2.25.1"
dask = "^2021.4.1"
fsspec = "^2021.4.0"
partd = "^1.2.0"
<private_package_name> = {git = https://<private_gitlab_repo>/<private_package>.git", rev = "master"}

[tool.poetry.extras]
dask = ["dataframe"]

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Issue

I have a package I'm using poetry to build and version. This private package has a dependency on another private package in a different repo. This other private package is not published to a package registry, so in order to access it, I've set up basic HTTP credentials to the git project directly (as recommended in https://github.com/python-poetry/poetry/pull/5567)

poetry config repositories.gitlab https://<private_gitlab_repo>/<private_package>.git
poetry config http-basic.gitlab <gitlab_deploy_token> <password>

This works fine when building/installing the package locally using poetry build and poetry install. However, when attempting to install my package as a dependency in an Airflow Docker image, the install of the private package fails due to missing authentication credentials. I'm using pip to install my package inside the Docker image, so it doesn't appear to have any notion of the Poetry credentials that were configured locally when building the package.

#12 84.81   Collecting <private_package_name>@ git+https://<private_gitlab_repo>/<private_package>.git@master
#12 84.81     Cloning https://<private_gitlab_repo>/<private_package>.git (to revision master) to /tmp/pip-resolver-gg2sgq6r/<private_package_name>
#12 84.82     Running command git clone --filter=blob:none -q https://<private_gitlab_repo>/<private_package>.git /tmp/pip-resolver-gg2sgq6r/<private_package_name>
#12 85.32     fatal: could not read Username for 'https://<private_gitlab_repo>': No such device or address
#12 85.32 Traceback (most recent call last):
#12 85.32   File "/home/airflow/.local/bin/pip-compile", line 8, in <module>
#12 85.32     sys.exit(cli())
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
#12 85.32     return self.main(*args, **kwargs)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
#12 85.32     rv = self.invoke(ctx)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
#12 85.32     return ctx.invoke(self.callback, **ctx.params)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
#12 85.32     return callback(*args, **kwargs)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
#12 85.32     return f(get_current_context(), *args, **kwargs)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/scripts/compile.py", line 487, in cli
#12 85.32     results = resolver.resolve(max_rounds=max_rounds)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/resolver.py", line 266, in resolve
#12 85.32     has_changed, best_matches = self._resolve_one_round()
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/resolver.py", line 356, in _resolve_one_round
#12 85.32     their_constraints.extend(self._iter_dependencies(best_match))
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/resolver.py", line 451, in _iter_dependencies
#12 85.32     dependencies = self.repository.get_dependencies(ireq)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/repositories/pypi.py", line 251, in get_dependencies
#12 85.32     download_dir, ireq, wheel_cache
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/piptools/repositories/pypi.py", line 213, in resolve_reqs
#12 85.32     results = resolver._resolve_one(reqset, ireq)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 379, in _resolve_one
#12 85.32     dist = self._get_dist_for(req_to_install)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 332, in _get_dist_for
#12 85.32     dist = self.preparer.prepare_linked_requirement(req)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
#12 85.32     return self._prepare_linked_requirement(req, parallel_builds)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 528, in _prepare_linked_requirement
#12 85.32     link, req.source_dir, self._download, self.download_dir, hashes
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 190, in unpack_url
#12 85.32     unpack_vcs_link(link, location)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 65, in unpack_vcs_link
#12 85.32     vcs_backend.unpack(location, url=hide_url(link.url))
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 598, in unpack
#12 85.32     self.obtain(location, url=url)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 512, in obtain
#12 85.32     self.fetch_new(dest, url, rev_options)
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/vcs/git.py", line 269, in fetch_new
#12 85.32     dest,
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 649, in run_command
#12 85.32     stdout_only=stdout_only,
#12 85.32   File "/home/airflow/.local/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 254, in call_subprocess
#12 85.32     raise InstallationSubprocessError(proc.returncode, command_desc)
#12 85.32 pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 128: git clone --filter=blob:none -q https://<private_gitlab_repo>/<private_package>.git /tmp/pip-resolver-gg2sgq6r/<private_package_name> Check the logs for full command output.

I've tried setting environment variables (POETRY_HTTP_BASIC_GITLAB_USERNAME/POETRY_HTTP_BASIC_GITLAB_PASSWORD) for my private repo inside the Dockerfile, but this seems to be ignored by pip. I've tried hardcoding the private repo deploy_token/password in the git URL inside pyproject.toml, but this results in an error saying the git URL is invalid when attempting to build the package

poetry build
Building <package_name> (0.0.0)

Invalid git url "https://<deploy_token>:<password>@<private_gitlab_repo>/<private_package>.git"

This appears somewhat related to https://github.com/python-poetry/poetry/issues/2062. While the pre-release fix (https://github.com/python-poetry/poetry/pull/5567) seems to address the issue of supporting deploy tokens when adding HTTP basic authentication credentials for private gitlab projects, the absence of these credentials in the dependency URL itself prevents pip from accessing the package.

Start with the pyproject.toml dependency declaration and the poetry build output, then reproduce installation with pip in the Docker image using the reported GitLab dependency. Trace how the dependency URL is represented for pip and compare it with Poetry's configured credentials. Done means the documented private GitLab dependency can be installed through pip without invalid URL errors or missing credentials.