Requests ignores HSTS if redirected to http:// version of site
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
Requests will follow an unencrypted http:// redirect from a site that implements HSTS.
Expected Result
I expected Requests to force https:// handling of all http:// URLs for a site which has advertised Strict-Transport-Security.
Actual Result
Requests followed a 301 redirect to Location: http://site and, since port 80 was blocked for that site, spun forever and timed out.
(Note, the fact that an HSTS site is redirecting to a closed and unencrypted port is a sin on the site's behalf, not being defended here, but it illustrates the HSTS handling very nicely. Requests times out because it tries to follow the http:// link. Chrome, Firefox, Edge, and Internet Explorer all ignore the http:// referral and rewrite it as https://, arriving at the target landing page. Chrome developer mode describes it as a "307 Internal Redirect" and "Non-Authoritative-Reason: HSTS").
Reproduction Steps
import requests
r = requests.get('https://olb.bsf.net', timeout=(3,15))
Site sends a 302 Redirect to /login, then a 301 Redirect to http://olb.bsf.net/login/. This last redirect to an unencrypted URL is what exposes the behavior. Since the site doesn't listen on port 80, this request will time out and throw an urllib3.exceptions.MaxRetryError error.
System Information
$ python -m requests.help
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "2.6"
},
"implementation": {
"name": "CPython",
"version": "3.6.9"
},
"platform": {
"release": "5.7.6-x86_64-linode136",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.18.4"
},
"system_ssl": {
"version": "1010100f"
},
"urllib3": {
"version": "1.22"
},
"using_pyopenssl": false
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided requests.get reproduction and trace how Requests handles the 302 and subsequent 301 Location: http:// redirect after the site advertises HSTS. Done means the redirect is handled as HTTPS rather than attempting the unencrypted URL, with the reproduction no longer timing out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100