4paradigm / 4paradigm/OpenMLDB

load data issues and error msg improvements

Aperta
#3,708 1 commento 0 reazioni 1 assegnatario Rivendicata da @vagetablechicken Vedi su GitHub
high-priority
Lingua principale
C++
Stelle
1.7k
Fork
331
Merge medio
12g 12h
PR unite (30g)
1

Descrizione

- CLI local load_mode issues(high-priority)
- error msg
- inconsistent with cluster mode

# All methods to load data
method | desc | col convert failure | col set failure |row build failure | put failure | whole failure
-|-|-|-|-|-|-
insert sql | sql to insert row in router | | MakeDefault recursion, hard to print row, just print row idx | print row idx | print row idx and status msg | failed rows peek
java(sdk&jdbc) prepared stmt | getInsertPreparedStmt, optimized insert(FlexibleRowBuilder) | - | SQLException col pos | SQLException no row hint | log status msg | - (executeBatch returns 0/1)
load data cluster| getInsertPreparedStmt, but many rows | - | same | same | log, no hint in exception | readable row in exception msg
load data local | sql_cluster_router.cc | translate col name, type, value | cvt and set | readable row in status msg | status msg | file & lineno with error msg
api server | | JsonReader parse is easy, json -> row will print hint(put-cvt col name, type, value;deployment-cvt col name, type,value;query.parameter-cvt col type, idx ) | the same place | just one row | status msg | status msg|
jdbc insert row(not recommend)| | |

One row insertion report col level failures?
Multi rows insertion report row idx, if user can get row easily
Spark insertion print failed row(readable), cuz user can't get row easily in spark way

TODO openmldb-import use prepared stmt instead of getInsertRow
local use new csv library to support escape, but it may still != cluster spark style.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

The issue describes inconsistencies in error messages and behavior across different data loading methods (CLI local load_mode, cluster mode, Java SDK/JDBC, API server). Focus on sql_cluster_router.cc for CLI local load_mode improvements. Need to understand the current error handling in each method, compare with cluster mode, and implement improvements for error messages and consistency. Start by examining the code paths for load data in sql_cluster_router.cc and the prepared statement implementations.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java, python, spark, sql
Ambito
backend-api-design, cli, data-engineering, databases
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.