cloudpipe / cloudpipe/cloudpickle

Private dispatch table not checked

Open
#437 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

I'm attempting to register a custom reducer with cloudpickle. In my use case, I need multiple `CloudPickler`'s to serialize an object with different behavior, so I can't override the class-level global dispatch table.

To demonstrate, assume I have the following class:

```
class CustomClass:
def __init__(self, *args):
self.x = len(args)

c = CustomClass()
```

I can define a custom reducer and register it with [a private dispatch table](https://docs.python.org/3/library/pickle.html#pickle.Pickler.dispatch_table):

```
def custom_reducer(obj):
return CustomClass, (0,)

io_buffer = io.BytesIO()
custom_pickler = pickle.Pickler(io_buffer)
custom_pickler.dispatch_table = {CustomClass: custom_reducer}
custom_pickler.dump(c)

io_buffer.seek(0)
c1 = pickle.load(io_buffer)
assert c1.x == 1
```

I can also prove that this doesn't affect the global dispatch table:

```
c2 = pickle.loads(pickle.dumps(c))
assert c2.x == 0
```

Unfortunately, when I try the same behavior with cloudpickle, it doesn't respect the private dispatch table:

```
io_buffer = io.BytesIO()
custom_pickler = cloudpickle.CloudPickler(io_buffer)
custom_pickler.dispatch_table = {CustomClass: custom_reducer}
custom_pickler.dump(c)

io_buffer.seek(0)
c3 = cloudpickle.load(io_buffer)
assert c3.x == 1 # assertion fails becase c3.x == 0
```

[(reference to how this was implemented in pickle)](https://bugs.python.org/issue14166)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.