parse_requirements fails on requirements.txt files with multiple line continuations.

Open
#2,422 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
build-system

Research direction

Inspect pkg_resources/init.py around the parse_requirements continuation handling. Reproduce the failure with the provided requirements.txt example and a setup.py command, then add regression coverage for multiple continuations. Done means hashed pip-tools requirements parse successfully and the setup.py command no longer fails.

Written by the indexing model from the issue text.

Description

pip files with hashes, as generated by for instance the pip-compile command in pip-tools, are currently not parseable by parse_requirements in pkg_resources/__init__.py. It looks like the issue is multiple line continuations, as for instance here:

certifi==2020.6.20 \
    --hash=sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3 \
    --hash=sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41 \
    # via sentry-sdk

I think this:

https://github.com/pypa/setuptools/blob/master/pkg_resources/__init__.py#L3074

should be

while line.endswith('\\'):
    ...

rather than if.

I may have missed some subtle reason why this is not already the case, but it solved the issue I was running into locally where python setup.py [any command] was failing because I had a requirements.txt with hashes generated by pip-tools. (I think it's a fairly unusual edge case because we also use pbr, which I think is what's triggering the parsing of the requirements to happen when calling setup.py).

I'd like to get a reading on how likely a patch is to be accepted before I write up the PR and tests, but I'd be happy to do so.

If it's not something that's likely to get in, I'll probably dump pbr as a dependency instead.

Dominant language
Python
Stars
2.9k
Forks
1.4k
Avg merge
1d 1h
Merged PRs (30d)
1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pypa/setuptools

All issues in pypa/setuptools

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.