IPv6 Zone Identifiers are not correctly parsed (still)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
Issue #5775 notes that IPv6 Zone Identifiers are not parsed correctly when the Zone ID is itself a valid percent-encoded character from the UNRESERVED_SET. That issue was incorrectly closed as a duplicate of #5126, in fact #5126 is to do with a different, resolved, bug in urllib. This ticket here is a duplicate of #5775 but commenting on that ticket is now locked.
IPv6 addresses can have the form fe80::1:2:3:4%zone where zone is any alphanumeric sequence and is platform-dependent. In order to address their use in URLs, where the %zone could be interpreted as a percent-escaped character, RFC6874 requires the % to be replaced with its own percent-escaped representation %25, e.g. http://[fe80::1:2:3:4%25zone.
In requests, this is not enough to protect the Zone ID in the URL, if the zone is a valid percent-escaped character from the UNRESERVED_SET. Specifically, url.py::_normalize_host removes the RFC6874 sequence and replaces it with a simple %, then the round trip through quote/unquote_reserved in utils.py::requote_uri called from PreparedRequest::prepare_url transforms the Zone ID in to the percent-escaped character anyway.
Note that doubly escaping the percent works, but the URL is then neither intuitive nor RFC-compliant.
Expected Result
requests.Request('GET', 'http://[fe80::1:2:3:4%61]').prepare().url -> Undefined/don't care (not RFC-compliant)
requests.Request('GET', 'http://[fe80::1:2:3:4%2561]').prepare().url -> 'http://[fe80::1:2:3:4%61]'
requests.Request('GET', 'http://[fe80::1:2:3:4%252561]').prepare().url -> 'http://[fe80::1:2:3:4%2561]'
Actual Result
requests.Request('GET', 'http://[fe80::1:2:3:4%61]').prepare().url -> 'http://[fe80::1:2:3:4a]'
requests.Request('GET', 'http://[fe80::1:2:3:4%2561]').prepare().url -> 'http://[fe80::1:2:3:4a]'
requests.Request('GET', 'http://[fe80::1:2:3:4%252561]').prepare().url -> 'http://[fe80::1:2:3:4%61]'
System Information
$ python -m requests.help
{
"chardet": {
"version": null
},
"charset_normalizer": {
"version": "2.1.0"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "3.3"
},
"implementation": {
"name": "CPython",
"version": "3.10.5"
},
"platform": {
"release": "22.1.0",
"system": "Darwin"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.28.1"
},
"system_ssl": {
"version": "1010111f"
},
"urllib3": {
"version": "1.26.12"
},
"using_charset_normalizer": true,
"using_pyopenssl": false
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with url.py::_normalize_host and utils.py::requote_uri, then follow how PreparedRequest::prepare_url processes IPv6 URLs. Reproduce the three examples from the issue and compare each prepared URL with the expected result. Done means the RFC6874 zone-identifier cases no longer collapse valid percent-encoded characters, with regression coverage for the stated behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100