cloudpipe / cloudpipe/cloudpickle

Decorator to upgrade methods to cloudpickle

Open
#434 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

This is a suggestion/feature request. The idea is to provide a decorator to upgrade any callable to use cloudpickle by default. This comes handy for example to use a locally defined function via any standard executor.

The syntax would be something like:

from cloudpickle import inline

@inline
def myfunc(...):
...

executor.submit(myfunc)

Below is some sample code to achieve the equivalent:

import pickle
import cloudpickle

class InlineWrapper:
"""
Function wrapper with __reduce__ based on cloudpickle.dumps
Makes the callable pickleable even if defined in __main__
"""

def __init__(self, func):
self.func = func

def __reduce__(self):
data = cloudpickle.dumps(self.func)
return pickle.loads, (data,)

def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)

def inline(func):
""" Decorator for InlineWrapper """

return InlineWrapper(func)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.