4paradigm / 4paradigm/OpenMLDB

spark load data(online&offline), the blank value in the data will become null

Aperta
#2,459 3 commenti 0 reazioni 1 assegnatario Rivendicata da @vagetablechicken Vedi su GitHub
bug high-priority storage-engine
Lingua principale
C++
Stelle
1.7k
Fork
331
Merge medio
12g 12h
PR unite (30g)
1

Descrizione

**Bug Description**
When using load data to import data in CSV format, the empty string in the data will become null
![bbc42822e5496834964e8270e7cdcbb7](https://user-images.githubusercontent.com/18275570/189319963-ea3fe480-2661-46a9-a7a1-14cf81a2d055.png)

select into result:
9667b640bdc199847a35f6e0654691bf

**Expected Behavior**

**Relation Case**
test_select_into_load_data.yaml id:0-1. 0-2、18-2、62

**Steps to Reproduce**
```sql
create table auto_YuMSwEvO(
id int,
c1 string,
c2 smallint,
c3 int,
c4 bigint,
c5 float,
c6 double,
c7 timestamp,
c8 date,
c9 bool,
index(key=(c1),ts=c7))options(partitionnum=1,replicanum=1);
insert into auto_YuMSwEvO values
(3,'',3,22,32,1.3,2.3,1590738991000,'2020-05-03',true);
create table auto_FwREKcjN(
id int,
c1 string,
c2 smallint,
c3 int,
c4 bigint,
c5 float,
c6 double,
c7 timestamp,
c8 date,
c9 bool,
index(key=(c1),ts=c7))options(partitionnum=1,replicanum=1);
set @@SESSION.execute_mode = "online";
select * from auto_YuMSwEvO into outfile '/Users/zhaowei/code/4paradigm/OpenMLDB/auto_YuMSwEvO.csv' ;
LOAD DATA INFILE 'file:///Users/zhaowei/code/4paradigm/OpenMLDB/auto_YuMSwEvO.csv' into table auto_FwREKcjN options(mode='append');
```

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

The issue involves CSV data loading where empty strings become NULL. Look at the Spark load data path in OpenMLDB, likely in the Spark connector or data import modules. Check the test case referenced (test_select_into_load_data.yaml) to understand the expected behavior. Reproduce the bug using the provided SQL steps, then examine how CSV parsing handles empty values versus nulls.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
spark, sql
Ambito
data-engineering, databases
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.