requests with pyOpenSSL is drastically slower than with stdlib's ssl module
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 944
- Forks
- 440
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 6
Description
Hello pyOpenSSL gurus!
We have a file transfer client (https://github.com/Azure/azure-data-lake-store-python) that relies on:
cffi
oathlib
requests
requests-oathlib
When we run performance benchmarks on this client with just those packages installed, we are seeing upload throughput around 7-8gbps.
When pyOpenSSL is installed (by a different package that depends on it) or upload throughput drops to no higher than 2gbps.
Based on the above info, I have a couple questions:
- How is pyOpenSSL impacting my package that does not take any dependency on it, and how can I prevent it from having an impact, if I can (that would be ideal!)?
- Is this a known issue in pyOpenSSL when submitting a lot of PUT requests to a server in parallel? Is there something I can do in my package to have it work better with pyOpenSSL in this scenario?
Please let me know if you need any other information from me or if you have any quick fixes that I can try out and thank you all so much for your time!
Ben
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Azure Data Lake Store client benchmark with cffi, oauthlib, requests, requests-oauthlib, and pyOpenSSL installed versus the stdlib ssl module. Compare parallel PUT upload throughput and review the issue discussion for diagnostic details; done requires identifying the source of the slowdown and an actionable mitigation or confirmed limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100