Combining optional prod package with mandatory dev extra causes prod package to be non-optional

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate. (#2868 is similar but not the same issue.)

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Arch Linux / unknown OS

• Poetry version: 1.1.7

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/l0b0/c0308091c7a748e6d481791c79a6bbda

Issue

In a project I'm working on we have a few different sets of production dependencies, one for each service. To avoid bloating each service with all the production packages we've marked all the production packages as optional and created extras for each of them, listing the relevant packages. The problem is when I want to introduce a mandatory dev dependency on an extra which is also an optional production dependency, like this:

[tool.poetry.dependencies]
python = "^3.9"
SQLAlchemy = {version = "*", optional = true}

[tool.poetry.dev-dependencies]
SQLAlchemy = {version = "*", extras = ["mypy"]}

[tool.poetry.extras]
foo = [
    "SQLAlchemy",
]

poetry lock in this case changes the sqlalchemy production package to be mandatory:

@@ -1327,7 +1327,7 @@ name = "sqlalchemy"
 version = "1.4.22"
 description = "Database Abstraction Library"
 category = "main"
-optional = true
+optional = false
 python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7"

The reason I'm reporting this is that Dependabot is having trouble with the original files:

updater | ERROR <job_187255052> Error processing sqlalchemy (Dependabot::SharedHelpers::HelperSubprocessFailed)
updater | ERROR <job_187255052> Creating virtualenv foo-OyE-KPju-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
updater | <job_187255052> Updating dependencies
updater | <job_187255052> Resolving dependencies...
updater | <job_187255052> 
updater | <job_187255052> Writing lock file
updater | <job_187255052> 
updater | <job_187255052>   AttributeError
updater | <job_187255052> 
updater | <job_187255052>   'EmptyConstraint' object has no attribute 'allows'
updater | <job_187255052> 
updater | <job_187255052>   at /usr/local/.pyenv/versions/3.9.6/lib/python3.9/site-packages/poetry/core/version/markers.py:291 in validate
updater | <job_187255052>       287│ 
updater | <job_187255052>       288│         if self._name not in environment:
updater | <job_187255052>       289│             return True
updater | <job_187255052>       290│ 
updater | <job_187255052>     → 291│         return self._constraint.allows(self._parser(environment[self._name]))
updater | <job_187255052>       292│ 
updater | <job_187255052>       293│     def without_extras(self):  # type: () -> MarkerTypes
updater | <job_187255052>       294│         return self.exclude("extra")
updater | <job_187255052>       295│
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker/poetry_version_resolver.rb:323:in `run_poetry_command'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker/poetry_version_resolver.rb:88:in `block (2 levels) in fetch_latest_resolvable_version_string'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/shared_helpers.rb:160:in `with_git_configured'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker/poetry_version_resolver.rb:76:in `block in fetch_latest_resolvable_version_string'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/shared_helpers.rb:46:in `block (2 levels) in in_a_temporary_directory'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/shared_helpers.rb:46:in `chdir'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/shared_helpers.rb:46:in `block in in_a_temporary_directory'
updater | ERROR <job_187255052> /usr/lib/ruby/2.7.0/tmpdir.rb:89:in `mktmpdir'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/shared_helpers.rb:43:in `in_a_temporary_directory'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker/poetry_version_resolver.rb:75:in `fetch_latest_resolvable_version_string'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker/poetry_version_resolver.rb:48:in `latest_resolvable_version'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-python-0.160.0/lib/dependabot/python/update_checker.rb:43:in `latest_resolvable_version'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/update_checkers/base.rb:74:in `preferred_resolvable_version'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/update_checkers/base.rb:257:in `preferred_version_resolvable_with_unlock?'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/update_checkers/base.rb:249:in `numeric_version_can_update?'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/update_checkers/base.rb:199:in `version_can_update?'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/vendor/ruby/2.7.0/gems/dependabot-common-0.160.0/lib/dependabot/update_checkers/base.rb:44:in `can_update?'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:429:in `requirements_to_unlock'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:224:in `check_and_create_pull_request'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:80:in `check_and_create_pr_with_error_handling'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:56:in `block in run'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:56:in `each'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:56:in `run'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/update_files_job.rb:17:in `perform_job'
updater | ERROR <job_187255052> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:28:in `run'
updater | ERROR <job_187255052> bin/update_files.rb:21:in `<main>'

A workaround is to split the definition in two, by figuring out the name of the packages corresponding to the SQL Alchemy "mypy" extra, in this case sqlalchemy2-stubs:

@@ -66,7 +66,7 @@ pylint = "*"
 pytest = "*"
 pytest-randomly = "*"
 pytest-subtests = "*"
-SQLAlchemy = {version = "*", optional = true, extras = ["mypy"]}
+sqlalchemy2-stubs = "*"
 types-python-dateutil = "*"
 types-setuptools = "*"
 types-six = "*"

Start by reproducing the provided pyproject.toml setup and running poetry lock, then compare the generated lock entry and the reported EmptyConstraint traceback. Done means the optional SQLAlchemy production package remains optional when its extra is used by a mandatory dev dependency, and locking no longer raises the reported error.