4paradigm / 4paradigm/OpenMLDB

load data issues and error msg improvements

Ouverte
#3,708 1 commentaire 0 réactions 1 personne assignée Réclamée par @vagetablechicken Voir sur GitHub
high-priority
Langage dominant
C++
Étoiles
1.7k
Forks
331
Merge moyen
12 j 12 h
PR mergées (30 j)
1

Description

- CLI local load_mode issues(high-priority)
- error msg
- inconsistent with cluster mode

# All methods to load data
method | desc | col convert failure | col set failure |row build failure | put failure | whole failure
-|-|-|-|-|-|-
insert sql | sql to insert row in router | | MakeDefault recursion, hard to print row, just print row idx | print row idx | print row idx and status msg | failed rows peek
java(sdk&jdbc) prepared stmt | getInsertPreparedStmt, optimized insert(FlexibleRowBuilder) | - | SQLException col pos | SQLException no row hint | log status msg | - (executeBatch returns 0/1)
load data cluster| getInsertPreparedStmt, but many rows | - | same | same | log, no hint in exception | readable row in exception msg
load data local | sql_cluster_router.cc | translate col name, type, value | cvt and set | readable row in status msg | status msg | file & lineno with error msg
api server | | JsonReader parse is easy, json -> row will print hint(put-cvt col name, type, value;deployment-cvt col name, type,value;query.parameter-cvt col type, idx ) | the same place | just one row | status msg | status msg|
jdbc insert row(not recommend)| | |

One row insertion report col level failures?
Multi rows insertion report row idx, if user can get row easily
Spark insertion print failed row(readable), cuz user can't get row easily in spark way

TODO openmldb-import use prepared stmt instead of getInsertRow
local use new csv library to support escape, but it may still != cluster spark style.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

The issue describes inconsistencies in error messages and behavior across different data loading methods (CLI local load_mode, cluster mode, Java SDK/JDBC, API server). Focus on sql_cluster_router.cc for CLI local load_mode improvements. Need to understand the current error handling in each method, compare with cluster mode, and implement improvements for error messages and consistency. Start by examining the code paths for load data in sql_cluster_router.cc and the prepared statement implementations.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
java, python, spark, sql
Domaine
backend-api-design, cli, data-engineering, databases
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
25/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.