DTStack / DTStack/chunjun

chunjun SQL模式 source源端SQL语法解析BUG

Open
#1,413 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar issues.

### What happened

1、kafka上原始数据:
{
"ver": "v2.2.2",
"pKey": "5949",
"sn": "03",
"ts": 16623128445200,
"devs": [
{
"dev": "41",
"sysId": "2001",
"d": [
{
"m": "WPP",
"v": 150
}
]
}
]
}

2、用SQL创建来源:
CREATE TABLE source_ods
(
ver STRING,
pKey STRING,
sn STRING,
ts STRING,
devs array< row >>>
) WITH (
'connector' = 'kafka-x'
,'topic' = 'xxxx'
,'properties.bootstrap.servers' = 'xxx'
,'properties.group.id' = 'dwd'
,'scan.startup.mode' = 'earliest-offset'
,'format' = 'json'
,'json.timestamp-format.standard' = 'SQL'
,'json.ignore-parse-errors' = 'true'
);

3、问题点:
chunjun建表JSON无法使用字符串接收
devs array< row >>>

但是我原生FLINK 是可以这样接收创建源端
devs array< row >

也可以这样接收创建源端
devs STRING

### What you expected to happen

1、kafka上原始数据:
{
"ver": "v2.2.2",
"pKey": "5949",
"sn": "03",
"ts": 16623128445200,
"devs": [
{
"dev": "41",
"sysId": "2001",
"d": [
{
"m": "WPP",
"v": 150
}
]
}
]
}

2、用SQL创建来源:
CREATE TABLE source_ods
(
ver STRING,
pKey STRING,
sn STRING,
ts STRING,
devs array< row >>>
) WITH (
'connector' = 'kafka-x'
,'topic' = 'xxxx'
,'properties.bootstrap.servers' = 'xxx'
,'properties.group.id' = 'dwd'
,'scan.startup.mode' = 'earliest-offset'
,'format' = 'json'
,'json.timestamp-format.standard' = 'SQL'
,'json.ignore-parse-errors' = 'true'
);

3、问题点:
chunjun建表JSON无法使用字符串接收
devs array< row >>>

但是我原生FLINK 是可以这样接收创建源端
devs array< row >

也可以这样接收创建源端
devs STRING

![1669683688(1)](https://user-images.githubusercontent.com/34857750/204412674-a34c2c72-7379-41e9-bcdf-0ba77a88d0d7.png)

### How to reproduce

1、kafka上原始数据:
{
"ver": "v2.2.2",
"pKey": "5949",
"sn": "03",
"ts": 16623128445200,
"devs": [
{
"dev": "41",
"sysId": "2001",
"d": [
{
"m": "WPP",
"v": 150
}
]
}
]
}

2、用SQL创建来源:
CREATE TABLE source_ods
(
ver STRING,
pKey STRING,
sn STRING,
ts STRING,
devs array< row >>>
) WITH (
'connector' = 'kafka-x'
,'topic' = 'xxxx'
,'properties.bootstrap.servers' = 'xxx'
,'properties.group.id' = 'dwd'
,'scan.startup.mode' = 'earliest-offset'
,'format' = 'json'
,'json.timestamp-format.standard' = 'SQL'
,'json.ignore-parse-errors' = 'true'
);

3、问题点:
chunjun建表JSON无法使用字符串接收
devs array< row >>>

但是我原生FLINK 是可以这样接收创建源端
devs array< row >

也可以这样接收创建源端
devs STRING

![1669683688(1)](https://user-images.githubusercontent.com/34857750/204412674-a34c2c72-7379-41e9-bcdf-0ba77a88d0d7.png)

### Anything else

1、kafka上原始数据:
{
"ver": "v2.2.2",
"pKey": "5949",
"sn": "03",
"ts": 16623128445200,
"devs": [
{
"dev": "41",
"sysId": "2001",
"d": [
{
"m": "WPP",
"v": 150
}
]
}
]
}

2、用SQL创建来源:
CREATE TABLE source_ods
(
ver STRING,
pKey STRING,
sn STRING,
ts STRING,
devs array< row >>>
) WITH (
'connector' = 'kafka-x'
,'topic' = 'xxxx'
,'properties.bootstrap.servers' = 'xxx'
,'properties.group.id' = 'dwd'
,'scan.startup.mode' = 'earliest-offset'
,'format' = 'json'
,'json.timestamp-format.standard' = 'SQL'
,'json.ignore-parse-errors' = 'true'
);

3、问题点:
chunjun建表JSON无法使用字符串接收
devs array< row >>>

但是我原生FLINK 是可以这样接收创建源端
devs array< row >

也可以这样接收创建源端
devs STRING

![1669683688(1)](https://user-images.githubusercontent.com/34857750/204412674-a34c2c72-7379-41e9-bcdf-0ba77a88d0d7.png)

### Version

1.12_release

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.