pyca / pyca/pyopenssl

requests with pyOpenSSL is drastically slower than with stdlib's ssl module

Open
#625 23 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
944
Forks
440
Avg merge
1d 6h
Merged PRs (30d)
6

Description

Hello pyOpenSSL gurus!

We have a file transfer client (https://github.com/Azure/azure-data-lake-store-python) that relies on:
cffi
oathlib
requests
requests-oathlib

When we run performance benchmarks on this client with just those packages installed, we are seeing upload throughput around 7-8gbps.
When pyOpenSSL is installed (by a different package that depends on it) or upload throughput drops to no higher than 2gbps.

Based on the above info, I have a couple questions:

  1. How is pyOpenSSL impacting my package that does not take any dependency on it, and how can I prevent it from having an impact, if I can (that would be ideal!)?
  2. Is this a known issue in pyOpenSSL when submitting a lot of PUT requests to a server in parallel? Is there something I can do in my package to have it work better with pyOpenSSL in this scenario?

Please let me know if you need any other information from me or if you have any quick fixes that I can try out and thank you all so much for your time!
Ben

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Azure Data Lake Store client benchmark with cffi, oauthlib, requests, requests-oauthlib, and pyOpenSSL installed versus the stdlib ssl module. Compare parallel PUT upload throughput and review the issue discussion for diagnostic details; done requires identifying the source of the slowdown and an actionable mitigation or confirmed limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.