psf / psf/requests

Requests ignores HSTS if redirected to http:// version of site

Open
#5,575 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

Requests will follow an unencrypted http:// redirect from a site that implements HSTS.

Expected Result

I expected Requests to force https:// handling of all http:// URLs for a site which has advertised Strict-Transport-Security.

Actual Result

Requests followed a 301 redirect to Location: http://site and, since port 80 was blocked for that site, spun forever and timed out.

(Note, the fact that an HSTS site is redirecting to a closed and unencrypted port is a sin on the site's behalf, not being defended here, but it illustrates the HSTS handling very nicely. Requests times out because it tries to follow the http:// link. Chrome, Firefox, Edge, and Internet Explorer all ignore the http:// referral and rewrite it as https://, arriving at the target landing page. Chrome developer mode describes it as a "307 Internal Redirect" and "Non-Authoritative-Reason: HSTS").

Reproduction Steps

import requests
r = requests.get('https://olb.bsf.net', timeout=(3,15))

Site sends a 302 Redirect to /login, then a 301 Redirect to http://olb.bsf.net/login/. This last redirect to an unencrypted URL is what exposes the behavior. Since the site doesn't listen on port 80, this request will time out and throw an urllib3.exceptions.MaxRetryError error.

System Information

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.9"
  },
  "platform": {
    "release": "5.7.6-x86_64-linode136",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "1010100f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided requests.get reproduction and trace how Requests handles the 302 and subsequent 301 Location: http:// redirect after the site advertises HSTS. Done means the redirect is handled as HTTPS rather than attempting the unencrypted URL, with the reproduction no longer timing out.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.