hazelcast / hazelcast/hazelcast-python-client
PythonObjectSerializer raise 'utf-8' codec can't decode byte
- Dominant language
- Python
- Stars
- 116
- Forks
- 78
- Avg merge
- 10d 22h
- Merged PRs (30d)
- 1
Description
Good time of the day team
When I tried to cache API responses, it failed as sometimes I have UnicodeDecodeError.
Looking deeper, I found that this is happening because of this code `out.write_string(cPickle.dumps(obj, 0).decode("utf-8"))` in `PythonObjectSerializer`
Full Code:
```python
class PythonObjectSerializer(BaseSerializer):
def read(self, inp):
str = inp.read_string().encode()
return cPickle.loads(str)
def write(self, out, obj):
out.write_string(cPickle.dumps(obj, 0).decode("utf-8"))
def get_type_id(self):
return PYTHON_TYPE_PICKLE
```
Issue example:
```console
>>> import pickle
>>> pickle.dumps("\u00e4").decode("utf-8")
Traceback (most recent call last):
File "", line 1, in
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
````
As a workaround, I created such a custom serializer:
```
class HazelcastJsonSerializer(StreamSerializer):
def read(self, inp):
return json.loads(inp.read_string())
def write(self, out, obj):
out.write_string(json.dumps(obj))
def get_type_id(self):
…
```
Is there any better solution?
python version: 3.6
Contributor guide
Assessment
This issue has not been assessed yet.