[C++][Java] Cross language incompatability when working with maps
- Dominant language
- C++
- Stars
- 26.5k
- Forks
- 3.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Summary
Java and C++ have different string compare logic resulting in incompatible binaries when sending between services that are built in different languages.
### Investigation
According to the internals documentation, the canonical comparison approach should be using `strcmp` as the comparison algorithm for strings.
https://flatbuffers.dev/internals/
The C++ implementation follows this logic according to its code generator
https://github.com/google/flatbuffers/blob/81edeb17d9118143f2c81caf27edfb0df401279e/src/idl_gen_cpp.cpp#L2608-L2610
The Java implementation does not follow this and implements their own algorithm
https://github.com/google/flatbuffers/blob/81edeb17d9118143f2c81caf27edfb0df401279e/java/src/main/java/com/google/flatbuffers/Table.java#L279-L296
As a result the binaries generated from one language will be incompatible with another.
### Reproduction
#### Summary
Because C++ compares strings using unsigned bytes (where 97 < 195), it sorts the map keys as ["a", "é"]. However, Java's Table.compareStrings() implementation uses signed bytes (where -61 < 97), meaning it expects the binary's keys to be sorted as ["é", "a"].
#### Schema:
```
table Entry {
key: string (key);
value: int;
}
table Map {
entries: [Entry];
}
root_type Map;
```
#### Data Encoded:
We are encoding a vector of Entry tables containing two elements:
```
{"key": "a", "value": 1}
{"key": "é", "value": 2}
```
#### C++ Binary
```hex
000000 0c 00 00 00 00 00 06 00 08 00 04 00 06 00 00 00 >................<
000010 04 00 00 00 02 00 00 00 08 00 00 00 20 00 00 00 >............ ...<
000020 ec ff ff ff 08 00 00 00 01 00 00 00 01 00 00 00 >................<
000030 61 00 00 00 08 00 0c 00 04 00 08 00 08 00 00 00 >a...............<
000040 08 00 00 00 02 00 00 00 02 00 00 00 c3 a9 00 00 >................<
000050
```
#### Java Binary
```hex
000000 0c 00 00 00 00 00 06 00 08 00 04 00 06 00 00 00 >................<
000010 04 00 00 00 02 00 00 00 1c 00 00 00 04 00 00 00 >................<
000020 f4 ff ff ff 1c 00 00 00 01 00 00 00 08 00 0c 00 >................<
000030 04 00 08 00 08 00 00 00 10 00 00 00 02 00 00 00 >................<
000040 01 00 00 00 61 00 00 00 02 00 00 00 c3 a9 00 00 >....a...........<
000050
```
Contributor guide
Research direction
Compare the string-ordering logic in src/idl_gen_cpp.cpp and java/src/main/java/com/google/flatbuffers/Table.java, starting with the linked implementations and internals documentation. Reproduce the map case with "a" and "é", then verify that C++ and Java use compatible ordering for cross-language binaries and add regression coverage where the project’s existing tests fit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java
- Domain
- backend-api-design, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100