DTStack / DTStack/chunjun

Oracle 批处理之后接 Logminer 出现数据错误问题

Open
#611 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
A clear and concise description of what the bug is.
SQL 模式下
配置脚本 想法:
Oracle 批处理之后无缝对接Logminer 功能

表信息
-- ----------------------------
-- Table structure for DEMO
-- ----------------------------
CREATE TABLE "FLINKUSER"."DEMO" (
"ID" NUMBER,
"TTIME" DATE
)

执行FlinkX SQL 脚本

-- ----------------------------
CREATE TABLE source
(
ID string,
TTIME DATE
) WITH (
'connector' = 'oracle-x',
'url' = 'jdbc:oracle:thin:@XXX:1521:xxx',
'table-name' = 'FLINKUSER.DEMO',
'username' = 'XXX',
'password' = 'XXX',
'scan.fetch-size' = '2000',
'scan.query-timeout' = '10'
);
CREATE TABLE sink
(
ID string,
TTIME DATE,
PRIMARY KEY (ID) NOT ENFORCED
) WITH (
'connector' = 'oracle-x',
'url' = 'jdbc:oracle:thin:@xxx:1521:xxx',
'table-name' = 'FLINKUSER.DEMO1',
'username' = 'xxx',
'password' = 'xxx',
'sink.buffer-flush.max-rows' = '2000'
);
insert into sink
select *
from source u;
CREATE TABLE source1
(
ID string,
TTIME DATE
) WITH (
'connector' = 'oraclelogminer-x'
,'url' = 'jdbc:oracle:thin:@xxx:1521:xxx'
,'username' = 'xxx'
,'password' = 'xxx'
,'cat' = 'insert,delete,update'
,'table' = 'FLINKUSER.DEMO'
,'read-position' = 'current'
);
insert into sink
select *
from source1 u1;
-- ----------------------------

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.
批转流 无缝对接
**Screenshots**
If applicable, add screenshots to help explain your problem.
往Demo 表插入数据
INSERT INTO "FLINKUSER"."DEMO" ("ID", "TTIME") VALUES ('101', TO_DATE('2022-02-25 11:21:39', 'SYYYY-MM-DD HH24:MI:SS'));
![image](https://user-images.githubusercontent.com/49392769/155649390-f7fe7091-ce43-4050-b8ae-39ea5dee64b3.png)

DEMO1表收到数据 但是 数据异常
![image](https://user-images.githubusercontent.com/49392769/155649565-92d568cc-d8f9-4d39-8eea-e7f9074d06d2.png)

TTIME 数据为 1970-01-01 13:17:28

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.