Cannot trace back urllib3.connection function invocation
- Dominant language
- Python
- Stars
- 693
- Forks
- 91
- PR merge metrics
- No merged PRs in 30d
Description
Hi guys,
I am using pshtt to scan `urc-31-6.od.nih.gov` to expose its invalid certificate. If you try and access the domain in Chrome you'll see that it is using a certificate that is issued to `extranet.sbir.nih.gov`, and the certificate's subject alt names are also different (`extranet.sbir.nih.gov` and `www.extranet.sbir.nih.gov`).
When I run pshtt on this domain, I am getting this error message PRIOR to the scan completing.
`ERROR:urllib3.connection:Certificate did not match expected hostname: urc-31-6.od.nih.gov. Certificate: {'subject': ((('commonName', 'extranet.sbir.nih.gov'),),), 'subjectAltName': [('DNS', 'extranet.sbir.nih.gov'), ('DNS', 'www.extranet.sbir.nih.gov')]}`
I traced back this error message to the `urllib3` package, and more specifically the `connection.py` [file](https://github.com/hbashton/urllib3/blob/3d2e632650e2d2d11b4ebd40d136bc40b2eb8cad/urllib3/connection.py). Scrolling all the way down in that file, you'll see the `_match_hostname` function, and that is where the above-mentioned is getting logged from.
What I am trying to understand is, how pshtt is invoking this function. The function takes in two parameters, `cert` and `asserted_hostname`, and I understand how pshtt is passing in the `asserted_hostname`, but how is it passing in the `cert` data?
In my code, after running pshtt, I am running some SSLyze methods to get additional data about the domain's certificate. SSLyze is unable to establish a connection with the domain `urc-31-6.od.nih.gov`, and as a result of this, I cannot extract any certificate data.
And from my understanding of the pshtt codebase, it is also using SSLyze to acquire data used for its scans. If my codebase cannot establish an SSLyze connection with the domain server to get the `cert` data, how is pshtt being able to pass in the `cert` data as a parameter to the urllib3.connection _matches_hostname function?
I would like to know because I am trying to see how I can capture this certificate data in my own codebase even when SSLyze connections are unsuccessful.
Thank you so much,
Contributor guide
Research direction
Start with urllib3/connection.py, especially _match_hostname, then locate pshtt’s HTTPS and SSLyze entry points to trace where certificate data originates. The work is complete when the certificate flow and the relationship between SSLyze failures and urllib3 hostname checks are documented clearly enough to explain how pshtt obtains the data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100