alibaba / alibaba/DataX

oracle迁移至ob,启动datax后报空指针:oceanbasev10writer.OceanBaseV10Writer$Job.prepare

Open
#1,343 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

配置文件:

{
"job": {
"setting": {
"speed": {
"channel": 4
},
"errorLimit": {
"record": 0,
"percentage": 0.1
}
},
"content": [
{
"reader": {
"name": "oraclereader",
"parameter": {
"username": "username",
"password": "XXX",
"column": [
"*"
],
"connection": [
{
"table": [
"tc_tbusinflag"
],
"jdbcUrl": ["jdbc:oracle:thin:@XXX:1521:ora11g"]
}
]
}
},

"writer": {
"name": "oceanbasev10writer",
"parameter": {
"obWriteMode": "insert",
"column": [
"*"
],
"preSql": [
""
],
"connection": [
{
"jdbcUrl": "||_dsc_ob10_dsc_||cluster:tenant||_dsc_ob10_dsc_||jdbc:oceanbase://XXX:2883/schema1?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&useUnicode=true&characterEncoding=gbk",
"table": [
"tc_tbusinflag"
]
}
],
"username": "sys",
"password": "XXX",
"writerThreadCount":10,
"batchSize": 1000,
"memstoreThreshold": "0.9"
}
}
}
]
}
}

日志:
2022-04-14 23:52:45.056 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-04-14 23:52:45.057 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (C_SYS_GROUP,C_REQ_CODE,C_CFM_CODE,C_INTERFACE_REQ_CODE,C_INTERFACE_CFM_CODE,VC_REQ_NAME,VC_CFM_NAME,VC_REQ_NAME_EN,VC_CFM_NAME_EN) VALUES(?,?,?,?,?,?,?,?,?)
], which jdbcUrl like:[||_dsc_ob10_dsc_||hsta:tboracle_gbk||_dsc_ob10_dsc_||jdbc:oceanbase://10.20.30.219:2883/pftcs?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&useUnicode=true&characterEncoding=gbk&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2022-04-14 23:52:45.058 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2022-04-14 23:52:45.058 [job-0] INFO JobContainer - DataX Reader.Job [oraclereader] do prepare work .
2022-04-14 23:52:45.058 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2022-04-14 23:52:45.059 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2022-04-14 23:52:45.059 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=hsta:tboracle_gbk:sys :url=jdbc:oceanbase://10.20.30.219:2883/pftcs?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&useUnicode=true&characterEncoding=gbk&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-04-14 23:52:45.069 [job-0] INFO DbUtils - value for query [show variables like 'version'] is [3.1.2]
2022-04-14 23:52:45.072 [job-0] ERROR JobContainer - Exception when job run
java.lang.NullPointerException: null
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Job.prepare(OceanBaseV10Writer.java:90) ~[oceanbasev10writer-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.prepareJobWriter(JobContainer.java:724) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.prepare(JobContainer.java:309) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:115) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:92) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:171) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:204) [datax-core-0.0.1-SNAPSHOT.jar:na]

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied job configuration and logs, then inspect OceanBaseV10Writer.java at line 90 and the prepare entry point. Reproduce the failure with the shown Oracle-to-OceanBase configuration, trace which value is null, and confirm that the job handles this case without the reported NullPointerException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.