apache / apache/beam

[Bug]: StrUtf8Coder in Python can't deserialize what's encoded with Java StringUtf8Coder

Open
#26,918 2 comments 0 reactions 0 assignees View on GitHub
bug cross-language java P2 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

I tried to use Python StrUtf8Coder to deserialize an string encoded with Java StringUtf8Coder but couldn't work. This is because beam is using different ways of implementation in Java and Python.

In Java, bytes length is encoded into the output bytes: https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StringUtf8Coder.java#L50 while in Python encoding is simply calling value.encode('utf-8'): https://github.com/apache/beam/blob/master/sdks/python/apache_beam/coders/coders.py#L426-L439

### Issue Priority

Priority: 3 (minor)

### Issue Components

- [X] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.