python / python/peps

PEP 554: Issues with data passing and pickle

Open
#3,309 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
reStructuredText
Stars
5k
Forks
1.8k
Avg merge
2d 4h
Merged PRs (30d)
25

Description

I believe the example program given in PEP 554: https://peps.python.org/pep-0554/#passing-objects-via-pickle has some issues. It's based on a zero value (b'\x00') being a data terminator, but my attempts to recreate this (using threads instead of interpreters) showed that pickle will insert zero values in the middle of the data stream:

Python 3.10.4 (main, May  7 2022, 16:23:10) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> pickle.dumps("PEP 554")
b'\x80\x04\x95\x0b\x00\x00\x00\x00\x00\x00\x00\x8c\x07PEP 554\x94.'

I think the goal here is to simply demonstrate passing pickled objects through the pipe. To that end, it might make a better example to change the reading portion to read a fixed, relatively large, buffer at once and then decode that. Something like:

    data = os.read(reader, 1024)
    obj = pickle.loads(data)
    do_something(obj)

This is clearly not a general purpose solution, but I think for an example like this, not solving all of the corner cases is OK.

Thanks for all of the tremendous work already put into this! This will be very cool!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open the “Passing Objects via Pickle” section of PEP 554 and inspect the example’s pipe-reading and termination logic. Verify the example against the shown pickle output, then update the example so its documented behavior matches the intended demonstration and confirm the revised snippet can load the pickled object.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.