psf / psf/requests

Mention pickling in the docs for creating custom Transport Adapters

Open
#6,255 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

While subclassing HTTPAdapter as well as Session the class variable __attrs__ needs to be appended with any custom instance variables. Otherwise, the instance breaks during pickling, which can happen in many different scenarios - for example, during Django's setUpTestData deep copying/in-memory data isolation. I think it would be nice if this is mentioned in the documentation.

There's a classic example of a custom adapter for setting a timeout value that is mentioned in multiple blog posts. Unfortunately, all of these examples are essentially broken because of a lack of information about pickling and __attrs__ class variable.

And yet the pattern of overriding __attrs__ exists in many open source libs, for example:

https://github.com/requests/toolbelt/blob/master/requests_toolbelt/adapters/ssl.py#L46
https://github.com/docker/docker-py/blob/main/docker/transport/unixconn.py#L57

If this pattern is so commonly used and needed, it might be a good idea to add it to the docs. Again, this is because so many people can stumble upon this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the documentation section covering custom Transport Adapters and review how subclassing HTTPAdapter or Session is currently explained. Document that custom instance variables require updating attrs for pickling, including the timeout-adapter pattern; done means readers are warned about the failure mode and the required adjustment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.