[Bug] RemoteTableQuery keySerializer should not include partition keys
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
When doing a lookup join to the remote query service (for primary key tables), we pass the partition keys, a bucket number, and a row to join to the existing table. Right now, serialization of the row fails because the serializer expected by the remote query service client assumes that the row includes partition fields, but we do not pass them (this is the same in LocalTableQuery, not a bug).
Let's change the key serializer here to avoid looking for partition keys.
### Compute Engine
Flink
### Minimal reproduce step
Use the remote query service to perform a lookup join from a flink row to a paimon table. The table must be partitioned, and the partitioned key and trimmed key must have different types (e.g. int vs. string). I'll include a test case in the PR that trips it up.
### What doesn't meet your expectations?
```
java.lang.ClassCastException: class org.apache.paimon.data.BinaryString cannot be cast to class java.lang.Integer
at org.apache.paimon.data.GenericRow.getInt(GenericRow.java:152)
at org.apache.paimon.data.serializer.InternalRowSerializer.toBinaryRow(InternalRowSerializer.java:123)
at org.apache.paimon.flink.query.RemoteTableQuery.lookup(RemoteTableQuery.java:73)
```
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in org.apache.paimon.flink.query.RemoteTableQuery.lookup and trace its use of InternalRowSerializer.toBinaryRow. Reproduce a remote lookup join against a partitioned primary-key table where partition and trimmed key types differ, then verify serialization no longer reads partition fields and the ClassCastException is gone.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100