Issue with Latin Characters in URL
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
Getting UTF-8 when attempting to get a URL with Latin characters in the URL.
Expected Result
<Response [200]>
Actual Result
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 668, in send
history = [resp for resp in gen] if allow_redirects else []
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 668, in
history = [resp for resp in gen] if allow_redirects else []
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 247, in resolve_redirects
**adapter_kwargs
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 681, in send
r._next = next(self.resolve_redirects(r, request, yield_requests=True, **kwargs))
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 149, in resolve_redirects
url = self.get_redirect_target(resp)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 115, in get_redirect_target
return to_native_string(location, 'utf8')
File "/usr/local/lib/python3.7/site-packages/requests/_internal_utils.py", line 25, in to_native_string
out = string.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 64: invalid continuation byte
Reproduction Steps
import requests
requests.get("http://www.janes.com/article/92450/south-korea-launches-philippine-navy-s-second-josé-rizal-class-frigate?from_rss=1")
System Information
$ python -m requests.help
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "2.7"
},
"implementation": {
"name": "CPython",
"version": "3.7.3"
},
"platform": {
"release": "19.0.0",
"system": "Darwin"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.22.0"
},
"system_ssl": {
"version": "1000212f"
},
"urllib3": {
"version": "1.23"
},
"using_pyopenssl": false
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the request in the issue first, then inspect the redirect flow in requests/sessions.py and decoding in requests/_internal_utils.py, as shown in the traceback. Trace how the Location value is handled when it contains Latin characters. Done means the reproduction no longer raises UnicodeDecodeError and the existing test suite passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100