4paradigm / 4paradigm/OpenMLDB

spark load data(online&offline), the blank value in the data will become null

Ouverte
#2,459 3 commentaires 0 réactions 1 personne assignée Réclamée par @vagetablechicken Voir sur GitHub
bug high-priority storage-engine
Langage dominant
C++
Étoiles
1.7k
Forks
331
Merge moyen
12 j 12 h
PR mergées (30 j)
1

Description

**Bug Description**
When using load data to import data in CSV format, the empty string in the data will become null
![bbc42822e5496834964e8270e7cdcbb7](https://user-images.githubusercontent.com/18275570/189319963-ea3fe480-2661-46a9-a7a1-14cf81a2d055.png)

select into result:
9667b640bdc199847a35f6e0654691bf

**Expected Behavior**

**Relation Case**
test_select_into_load_data.yaml id:0-1. 0-2、18-2、62

**Steps to Reproduce**
```sql
create table auto_YuMSwEvO(
id int,
c1 string,
c2 smallint,
c3 int,
c4 bigint,
c5 float,
c6 double,
c7 timestamp,
c8 date,
c9 bool,
index(key=(c1),ts=c7))options(partitionnum=1,replicanum=1);
insert into auto_YuMSwEvO values
(3,'',3,22,32,1.3,2.3,1590738991000,'2020-05-03',true);
create table auto_FwREKcjN(
id int,
c1 string,
c2 smallint,
c3 int,
c4 bigint,
c5 float,
c6 double,
c7 timestamp,
c8 date,
c9 bool,
index(key=(c1),ts=c7))options(partitionnum=1,replicanum=1);
set @@SESSION.execute_mode = "online";
select * from auto_YuMSwEvO into outfile '/Users/zhaowei/code/4paradigm/OpenMLDB/auto_YuMSwEvO.csv' ;
LOAD DATA INFILE 'file:///Users/zhaowei/code/4paradigm/OpenMLDB/auto_YuMSwEvO.csv' into table auto_FwREKcjN options(mode='append');
```

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

The issue involves CSV data loading where empty strings become NULL. Look at the Spark load data path in OpenMLDB, likely in the Spark connector or data import modules. Check the test case referenced (test_select_into_load_data.yaml) to understand the expected behavior. Reproduce the bug using the provided SQL steps, then examine how CSV parsing handles empty values versus nulls.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
spark, sql
Domaine
data-engineering, databases
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
45/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.