python / python/cpython

Improve performance of deepcopy by setting keep alive key in the memo

Open
#123,745 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

In the deepcopy implementation an exception is generated when the memo dict does not contain a key id(memo). We can avoid generating an exception by adding this key when the memo is generated. As a consequence the _keep_alive method is just a single line of code and we can inline this.

There is a behavior change: a call deepcopy(x, {}) works with current main, but will raise an exception (since the key id(memo)) has not been set. This might be acceptable as the documentation states "The memo dictionary should be treated as an opaque object." (see https://docs.python.org/3/library/copy.html). And in the code itself the memo argument is not documented at all https://github.com/python/cpython/blob/9aea9c100fbbaa1437f595da65417204872106a9/Lib/copy.py#L3-L9

There are some unit tests involving the memo, but those seem to test the behavior of the deepcopy and not of the memo argument.

If the behavior change is not acceptable because memo is part of the public interface, should we then instead document the structure of the argument memo` as being an implementation detail?

The argument _nil as a sentinel to the deepcopy is redundant, as the memo can only contain integers so we can use None instead.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-123746

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading Lib/copy.py and the existing unit tests that exercise deepcopy with a memo. Review linked PR gh-123746 and resolve whether the memo behavior is a public-interface change; done means the chosen implementation, tests, and any needed documentation consistently define the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.