psf / psf/requests

requests.Session doesn't properly handle closed keep-alive sessions

Open
#4,664 18 comments 15 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

Hello,
this was reported for requests 2.18.4 on Debian BTS[¹] by Jonathan Lynch, since it's not Debian specific I'm forwarding here:

When a server reaps a keep-alive session it sends a FIN packet to the
client. Normally, requests handles this fine and rebuilds the session on
the next request. However, there is an edge case involving network latency
that is not properly handled:

If python sends a request at roughly the same time as the server closes the
session, then the server will send a RST (as the session is closed). Python
receives this RST on what it thought was a valid session and throws an
error:

requests.exceptions.ConnectionError: ('Connection aborted.',
RemoteDisconnected('Remote end closed connection without response',))

The reason I consider this a bug is because python received the FIN packet
before it received the RST. As a result, it shouldn't be surprised when the
connection is subsequently aborted. It is an edge case, but the client has
enough information available to it that it could have handled it correctly.

The workaround is to set max_retries on the Session via an HTTPAdaptor, but
I believe the correct behavior when the FIN is received is to rebuild the
session and re-send any requests that were in-flight (rather than throwing
an error). Requests correctly handles the FIN packet if there are no
in-flight requests, but if there are in-flight requests it ignores it and
instead throws an error.

I will ask Jonathan to continue the discussion here. Thanks!

[¹] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=899406

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from requests.Session and the HTTPAdaptor retry configuration described in the issue, then trace how a keep-alive connection is reused after a FIN and how an in-flight request handles the subsequent RST. Reproduce the closed-session race if possible and verify that the request is retried on a rebuilt session instead of raising ConnectionError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.