_pickle.UnpicklingError: Memo value not found at index 1
Open
Nobody has claimed this yet.
3.12
3.13
3.14
docs
stdlib
type-bug
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
When pickling a container object. The pickler class will store child-objects to its self.memo even if the container could not be fully pickled (and not written to the output).
This can cause references to memo objects that are not in the output stream.
import sys
import pickle
import io
buf = io.BytesIO()
pickler = pickle.Pickler(buf)
i_will_be_memorized = 'qqqqqqqqqqqq'
i_cant_be_pickled = sys
try:
pickler.dump([i_will_be_memorized, i_cant_be_pickled])
except TypeError:
# i_will_be_memorized is now in pickler memory despite the exception
pass
# this dump will use the memo
pickler.dump(i_will_be_memorized)
data = buf.getbuffer().tobytes()
# so this will fail with:
# _pickle.UnpicklingError: Memo value not found at index 1
pickle.loads(data)
CPython versions tested on:
3.12
Operating systems tested on:
Windows
Linked PRs
- gh-125762
- gh-130231
- gh-130232
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the Python pickle and io.BytesIO snippet in the issue, then inspect pickle.Pickler's memo handling around a failed container dump. Done means a later dump does not reference memo entries absent from the output stream, with regression coverage for the demonstrated sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100