4paradigm / 4paradigm/OpenMLDB
load data issues and error msg improvements
- 主要语言
- C++
- 星标
- 1.7k
- 派生
- 331
- 平均合并
- 12 天 12 小时
- 30 天内合并 PR
- 1
描述
- CLI local load_mode issues(high-priority)
- error msg
- inconsistent with cluster mode
# All methods to load data
method | desc | col convert failure | col set failure |row build failure | put failure | whole failure
-|-|-|-|-|-|-
insert sql | sql to insert row in router | | MakeDefault recursion, hard to print row, just print row idx | print row idx | print row idx and status msg | failed rows peek
java(sdk&jdbc) prepared stmt | getInsertPreparedStmt, optimized insert(FlexibleRowBuilder) | - | SQLException col pos | SQLException no row hint | log status msg | - (executeBatch returns 0/1)
load data cluster| getInsertPreparedStmt, but many rows | - | same | same | log, no hint in exception | readable row in exception msg
load data local | sql_cluster_router.cc | translate col name, type, value | cvt and set | readable row in status msg | status msg | file & lineno with error msg
api server | | JsonReader parse is easy, json -> row will print hint(put-cvt col name, type, value;deployment-cvt col name, type,value;query.parameter-cvt col type, idx ) | the same place | just one row | status msg | status msg|
jdbc insert row(not recommend)| | |
One row insertion report col level failures?
Multi rows insertion report row idx, if user can get row easily
Spark insertion print failed row(readable), cuz user can't get row easily in spark way
TODO openmldb-import use prepared stmt instead of getInsertRow
local use new csv library to support escape, but it may still != cluster spark style.
贡献指南
调研方向
The issue describes inconsistencies in error messages and behavior across different data loading methods (CLI local load_mode, cluster mode, Java SDK/JDBC, API server). Focus on sql_cluster_router.cc for CLI local load_mode improvements. Need to understand the current error handling in each method, compare with cluster mode, and implement improvements for error messages and consistency. Start by examining the code paths for load data in sql_cluster_router.cc and the prepared statement implementations.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java, python, spark, sql
- 领域
- backend-api-design, cli, data-engineering, databases
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100