DTStack / DTStack/chunjun

[Bug] [sqlservercdcreader] mssqlcdc_stream2.json如何有转换语句就获取不了数据

Open
#1,440 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar issues.

### What happened

![image](https://user-images.githubusercontent.com/118407740/206344159-8124304d-97bf-4fd1-85b5-73d5384e05da.png)

### What you expected to happen

local运行模式,json格式脚本,sqlserver-cdc->stream,如果带有transformer语句,就不能正常获取数据,都是空的;如把transformer语句去掉,则正常。

### How to reproduce

一、环境准备
1、sqlserver版本信息
Microsoft SQL Server 2017 (RTM-CU21) (KB4557397) - 14.0.3335.7 (X64)
Jun 12 2020 20:39:00
Copyright (C) 2017 Microsoft Corporation
Enterprise Edition (64-bit) on Linux (CentOS Linux 7 (Core))
2、源端表结构信息
![image](https://user-images.githubusercontent.com/118407740/206344445-271207f5-3b78-4347-bbb4-5bf789276fe8.png)

二、配置脚本
[root@t-hadoop01 sql-script]# cat mssqlcdc_stream2.json
{
"job" : {
"content" : [ {
"reader" : {
"parameter" : {
"databaseName":"GISS_TEST",
"tableList": ["dbo.MKT_CITY"],
"username" : "sa",
"password" : "paawd",
"cat" : "insert,delete,update",
"url" : "jdbc:sqlserver://ip:1433;databaseName=GISS_TEST",
"splitUpdate" : true,
"pavingData" : true,
"pollInterval" : 1000,
"column": [
{
"name": "id",
"type": "varchar"
},
{
"name": "name",
"type": "varchar"
},
{
"name": "del_flag",
"type": "int"
},
{
"name": "create_user",
"type": "varchar"
},
{
"name": "create_date",
"type": "datetime"
},
{
"name": "update_user",
"type": "varchar"
},
{
"name": "update_date",
"type": "datetime"
}
]
},
"table": {
"tableName": "sourceTable"
},
"name" : "sqlservercdcreader"
},
"transformer": {
"transformSql": "select id,name,del_flag,create_user,create_date,update_user,update_date from sourceTable"
},
"writer" : {
"parameter" : {
"print" : true,
"column": [
{
"name": "id",
"type": "int"
},
{
"name": "name",
"type": "varchar"
},
{
"name": "del_flag",
"type": "varchar"
},
{
"name": "create_user",
"type": "varchar"
},
{
"name": "create_date",
"type": "datetime"
},
{
"name": "update_user",
"type": "varchar"
},
{
"name": "update_date",
"type": "datetime"
}
]
},
"table": {
"tableName": "sinkTable"
},
"name" : "streamwriter"
}
} ],
"setting" : {
"speed" : {
"bytes" : 0,
"channel" : 1
}
}
}
}

三、执行命令
[root@t-hadoop01 chunjun-dist112]# sh bin/chunjun-local.sh -job sql-script/mssqlcdc_stream2.json

四、执行结果
1、源端执行插入语句
insert into dbo.mkt_city(id,name,del_flag,create_user,create_date,update_user,update_date)
values('35','雅安35',0,'admin','2022-12-07 08:17:00','admin','2022-12-07 08:17:00');
2、显示结果
![image](https://user-images.githubusercontent.com/118407740/206344978-d226c397-9dab-4431-836d-c5e086ccf499.png)

### Anything else

_No response_

### Version

1.12_release

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.