[Bug] IndexOutOfBoundsException when lookup join sql
- Lingua principale
- Java
- Stelle
- 3.4k
- Fork
- 1.4k
- Merge medio
- 1g 11h
- PR unite (30g)
- 396
Descrizione
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
paimon 1.1.1 +flink 1.18
Basically, all the trigger conditions are met:
1. DEBUG logging was enabled for org.apache.paimon.flink.lookup.FileStoreLookupFunction in the production environment.
2. A lookup join is being used, and the table created is a SecondaryIndexLookupTable.
3. The join key is contract_no, which is not the complete primary key. The logs show _KEY_contract_no and _KEY_reg_valid, indicating that the full key also involves reg_valid.
4. There is a mismatch in the field order among the lookup projection, Paimon's internal read schema, and the original table schema.
5. The DEBUG branch attempts to print the matched rows, which leads to the execution of logRow().
LookupJoin[8] switched from RUNNING to FAILED
java.lang.RuntimeException: java.lang.IndexOutOfBoundsException
at org.apache.paimon.flink.lookup.FileStoreLookupFunction.lookup
...
Caused by: java.lang.IndexOutOfBoundsException
at org.apache.paimon.memory.MemorySegment.get
at org.apache.paimon.memory.MemorySegmentUtils.copyToBytes
at org.apache.paimon.data.BinaryString.toString
at org.apache.paimon.flink.lookup.FileStoreLookupFunction.logRow
at org.apache.paimon.flink.lookup.FileStoreLookupFunction.lambda$lookupInternal$4
### Compute Engine
flink 1.18
### Minimal reproduce step
1. DEBUG logging was enabled for org.apache.paimon.flink.lookup.FileStoreLookupFunction in the production environment.
2. A lookup join is being used, and the table created is a SecondaryIndexLookupTable.
3. The join key is contract_no, which is not the complete primary key. The logs show _KEY_contract_no and _KEY_reg_valid, indicating that the full key also involves reg_valid.
4. There is a mismatch in the field order among the lookup projection, Paimon's internal read schema, and the original table schema.
5. The DEBUG branch attempts to print the matched rows, which leads to the execution of logRow().
### What doesn't meet your expectations?
none
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Parti da FileStoreLookupFunction.lookup e segui lookupInternal fino al percorso di logging DEBUG che chiama logRow(). Riproduci il problema con un lookup join di Flink 1.18 su una SecondaryIndexLookupTable, una chiave contract_no parziale e il logging DEBUG abilitato. Il lavoro è completato quando il lookup non fallisce più con IndexOutOfBoundsException durante il logging delle righe corrispondenti.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- java
- Ambito
- data-engineering, databases
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Attiva
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 55/100