4paradigm / 4paradigm/OpenMLDB

The discrete feature values in the gcformat sample data generated by the OpenMLDB SQL feature extraction script are inconsistent with those calculated by the PICO script

Open
#3,923 1 comment 0 reactions 1 assignee Assigned to @wyl4pd View on GitHub
bug
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

**Bug Description**
Service Version: 0.9.0
The discrete feature values in the gcformat sample data generated by the OpenMLDB SQL feature extraction script are inconsistent with those calculated by the PICO script.

**Expected Behavior**
Current incorrect format: label| slot:sign:origin-value
Correct format: label index| slot:sign:origin-value

**Relation Case**
OpenMLDB SQL Feature Extraction Example:
```
0| 1:0:1 2:4599670039981440374 3:6365000770384461703 4:0:93.200000
1| 1:0:2 2:5613161932270271752 3:-1384602352766124944 4:0:93.075000
0| 1:0:3 2:4599670039981440374 3:-6239076729344379818 4:0:92.893000
```
PICO Feature Extraction Example:
```
0 0| 2:-8773247204422130117:1 3:4042412524814531440 4:6048373541161169225 5:4681710344575317709:0x1.74ccccccccccdp6
1 1| 2:-8773247204422130117:2 3:6142047291687075953 4:1461111459061395210 5:4681710344575317709:0x1.744cccccccccdp6
0 2| 2:-8773247204422130117:3 3:4042412524814531440 4:3353218529862650678 5:4681710344575317709:0x1.73926e978d4fep6
```

**Steps to Reproduce**
1. data schema:
```
id[Int],age[Int],job[String],cons_price_idx[Double],y[Int]
```
2. PICO Feature Extraction Script:
```
target_y = binary_label(y)
f_id = continuous(id)
f_age = discrete(age)
f_job = discrete(job)
f_cons_price_idx = continuous(cons_price_idx)
```
4. OpenMLDB SQL Feature Extraction Script:
```
select gcformat(
binary_label(bool(y)),
continuous(id),
discrete(age),
discrete(job),
continuous(cons_price_idx)
) as instance from main_table
```

Contributor guide

Open the contributing guide

Research direction

The issue is about the gcformat output in OpenMLDB SQL. Examine the SQL feature extraction script and the underlying C++ implementation of the gcformat function. Compare the output format with the PICO script's expected format. The fix likely involves modifying the string formatting logic for discrete features to include the label index. Look for tests related to gcformat or feature extraction to validate the change.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.