Can't (de)serialize NamedTuple objects
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**What happened**: dask raises `KeyError` exception when de-serializing named tuple objects.
**What you expected to happen**: no errors, since named tuples are basically tuple objects.
**Minimal Complete Verifiable Example**:
```python
from dask.distributed.protocol import pickle
from typing import NamedTuple
class Record(NamedTuple):
value: int
pickle.loads(pickle.dumps(Record(10)))
```
**Anything else we need to know?**: native Python pickle module seems to serialize named tuple objects in a few bytes, for instance:
```python
import pickle
from typing import NamedTuple
class Record(NamedTuple):
value: int
print(pickle.dumps(Record(10))) # b'\x80\x04\x95\x1d\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x06Record\x94\x93\x94K\n\x85\x94\x81\x94.'
```
while dask's pickle prints:
```python
b'\x80\x05\x95\xc2\n\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94(\x8c\x08builtins\x94\x8c\x04type\x94\x93\x94\x8c\x06Record\x94\x8c\x06typing\x94\x8c\nNamedTuple\x94\x93\x94\x85\x94}\x94\x8c\t__slots__\x94)s\x8c f8e43232932d437ea7e37287c1a7c49f\x94Nt\x94R\x94\x8c\x1ccloudpickle.cloudpickle_fast\x94\x8c\x0f_class_setstate\x94\x93\x94h\x0f}\x94(\x8c\x07__doc__\x94\x8c\x0eRecord(value,)\x94h\x0c)\x8c\x07_fields\x94\x8c\x05value\x94\x85\x94\x8c\x0f_field_defaults\x94}\x94\x8c\x07__new__\x94h\x03\x8c\x0cstaticmethod\x94\x93\x94h\x00\x8c\r_builtin_type\x94\x93\x94\x8c\nLambdaType\x94\x85\x94R\x94(h\x1f\x8c\x08CodeType\x94\x85\x94R\x94(K\x02K\x00K\x00K\x02K\x03KCC\x0ct\x00|\x00|\x01f\x01\x83\x02S\x00\x94N\x85\x94\x8c\n_tuple_new\x94\x85\x94\x8c\x04_cls\x94h\x17\x86\x94\x8c\x08\x94\x8c\x08\x94K\x01C\x02\x0c\x00\x94))t\x94R\x94}\x94\x8c\x08__name__\x94\x8c\x11namedtuple_Record\x94sNNNt\x94R\x94h\x10\x8c\x12_function_setstate\x94\x93\x94h5}\x94}\x94(h2h\x1b\x8c\x0c__qualname__\x94\x8c\x0eRecord.__new__\x94\x8c\x0f__annotations__\x94}\x94h\x17h\x03\x8c\x03int\x94\x93\x94s\x8c\x0e__kwdefaults__\x94N\x8c\x0c__defaults__\x94)\x8c\n__module__\x94h3h\x14\x8c%Create new instance of Record(value,)\x94\x8c\x0b__closure__\x94N\x8c\x17_cloudpickle_submodules\x94]\x94\x8c\x0b__globals__\x94}\x94h(\x8c\x08builtins\x94\x8c\x07getattr\x94\x93\x94h\x03\x8c\x05tuple\x94\x93\x94\x8c\x07__new__\x94\x86\x94R\x94su\x86\x94\x86R0\x85\x94R\x94\x8c\x05_make\x94h\x03\x8c\x0bclassmethod\x94\x93\x94h"(h%(K\x02K\x00K\x00K\x03K\x06K\x13C2\x88\x02|\x00|\x01\x83\x02}\x02\x88\x00|\x02\x83\x01\x88\x01k\x03r\x17t\x00d\x01\x88\x01\x9b\x00d\x02t\x01|\x02\x83\x01\x9b\x00\x9d\x04\x83\x01\x82\x01|\x02S\x00\x94N\x8c\tExpected \x94\x8c\x10 arguments, got \x94\x87\x94\x8c\tTypeError\x94\x8c\x03len\x94\x86\x94\x8c\x03cls\x94\x8c\x08iterable\x94\x8c\x06result\x94\x87\x94\x8c+/usr/lib/python3.10/collections/__init__.py\x94hTM\xa4\x01C\x08\n\x02\x0c\x01\x18\x01\x04\x01\x94\x8c\x04_len\x94\x8c\nnum_fields\x94\x8c\ttuple_new\x94\x87\x94)t\x94R\x94}\x94(\x8c\x0b__package__\x94\x8c\x0bcollections\x94h2hl\x8c\x08__path__\x94]\x94\x8c\x1f/usr/lib/python3.10/collections\x94a\x8c\x08__file__\x94\x8c+/usr/lib/python3.10/collections/__init__.py\x94uNNh\x00\x8c\x10_make_empty_cell\x94\x93\x94)R\x94hs)R\x94hs)R\x94\x87\x94t\x94R\x94h7hy}\x94}\x94(h2hTh:\x8c\x0cRecord._make\x94h<}\x94h@NhANhBhlh\x14\x8c4Make a new Record object from a sequence or iterable\x94hDh\x00\x8c\n_make_cell\x94\x93\x94hI\x8c\x03len\x94\x93\x94\x85\x94R\x94h\x80K\x01\x85\x94R\x94h\x80hP\x85\x94R\x94\x87\x94hE]\x94hG}\x94u\x86\x94\x86R0\x85\x94R\x94\x8c\x08_replace\x94h"(h%(K\x01K\x01K\x00K\x03K\x06K\x1bC.|\x00\xa0\x00\x88\x00|\x01j\x01\x88\x01|\x00\x83\x03\xa1\x01}\x02|\x01r\x15t\x02d\x01t\x03|\x01\x83\x01\x9b\x02\x9d\x02\x83\x01\x82\x01|\x02S\x00\x94N\x8c\x1cGot unexpected field names: \x94\x86\x94(hT\x8c\x03pop\x94\x8c\nValueError\x94\x8c\x04list\x94t\x94\x8c\x04self\x94\x8c\x04kwds\x94h`\x87\x94hbh\x8fM\xae\x01C\x08\x14\x01\x04\x01\x12\x01\x04\x01\x94\x8c\x04_map\x94\x8c\x0bfield_names\x94\x86\x94)t\x94R\x94hjNNhs)R\x94hs)R\x94\x86\x94t\x94R\x94h7h\xa4}\x94}\x94(h2h\x8fh:\x8c\x0fRecord._replace\x94h<}\x94h@NhANhBhlh\x14\x8cEReturn a new Record object replacing specified fields with new values\x94hDh\x80h\x03\x8c\x03map\x94\x93\x94\x85\x94R\x94h\x80h\x18\x85\x94R\x94\x86\x94hE]\x94hG}\x94u\x86\x94\x86R0\x8c\x08__repr__\x94h"(h%(K\x01K\x00K\x00K\x01K\x03K\x13C\x10|\x00j\x00j\x01\x88\x00|\x00\x16\x00\x17\x00S\x00\x94\x8c/Return a nicely formatted representation string\x94\x85\x94\x8c\t__class__\x94h2\x86\x94h\x97\x85\x94hbh\xb4M\xb7\x01C\x02\x10\x02\x94\x8c\x08repr_fmt\x94\x85\x94)t\x94R\x94hjNNhs)R\x94\x85\x94t\x94R\x94h7h\xc3}\x94}\x94(h2h\xb4h:\x8c\x0fRecord.__repr__\x94h<}\x94h@NhANhBhlh\x14h\xb6hDh\x80\x8c\n(value=%r)\x94\x85\x94R\x94\x85\x94hE]\x94hG}\x94u\x86\x94\x86R0\x8c\x07_asdict\x94h"(h%(K\x01K\x00K\x00K\x01K\x04K\x13C\x10\x88\x00\x88\x01|\x00j\x00|\x00\x83\x02\x83\x01S\x00\x94\x8c9Return a new dict which maps field names to their values.\x94\x85\x94h\x16\x85\x94h\xbahbh\xcfM\xbb\x01h\xbb\x8c\x05_dict\x94\x8c\x04_zip\x94\x86\x94)t\x94R\x94hjNNhs)R\x94hs)R\x94\x86\x94t\x94R\x94h7h\xdd}\x94}\x94(h2h\xcfh:\x8c\x0eRecord._asdict\x94h<}\x94h@NhANhBhlh\x14h\xd1hDh\x80h\x03\x8c\x04dict\x94\x93\x94\x85\x94R\x94h\x80h\x03\x8c\x03zip\x94\x93\x94\x85\x94R\x94\x86\x94hE]\x94hG}\x94u\x86\x94\x86R0\x8c\x0e__getnewargs__\x94h"(h%(K\x01K\x00K\x00K\x01K\x02K\x13C\x08\x88\x00|\x00\x83\x01S\x00\x94\x8c7Return self as a plain tuple. Used by copy and pickle.\x94\x85\x94)h\xbahbh\xeeM\xbf\x01C\x02\x08\x02\x94\x8c\x06_tuple\x94\x85\x94)t\x94R\x94hjNNhs)R\x94\x85\x94t\x94R\x94h7h\xfa}\x94}\x94(h2h\xeeh:\x8c\x15Record.__getnewargs__\x94h<}\x94h@NhANhBhlh\x14h\xf0hDh\x80hM\x85\x94R\x94\x85\x94hE]\x94hG}\x94u\x86\x94\x86R0\x8c\x0e__match_args__\x94h\x18h\x17\x8c\x0c_collections\x94\x8c\x0c_tuplegetter\x94\x93\x94K\x00\x8c\x18Alias for field number 0\x94\x86\x94R\x94hB\x8c\x08__main__\x94h
Contributor guide
Assessment
This issue has not been assessed yet.