master-flink1.16.1:报 JdbcDynamicTableFactory 问题,sql脚本执行报错。
- Dominant language
- Java
- Stars
- 4.1k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Search before asking
- [ ] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar question.
- [ ] I had googled my question but i didn't get any help.
- [ ] I had read the documentation: [ChunJun doc](https://dtstack.github.io/chunjun) but it didn't help me.
### Description
问题是chunjun1.28 版本能执行的脚本,master-flink1.16.1,执行报错
2023-02-21 16:55:20.781 [main] INFO com.dtstack.chunjun.util.PluginUtil - ChunJun executionMode: standalone
2023-02-21 16:55:20.781 [main] INFO com.dtstack.chunjun.util.PluginUtil - ChunJun reset pipeline.jars: []
2023-02-21 16:55:20.989 [main] INFO com.dtstack.chunjun.util.PluginUtil - ChunJun executionMode: standalone
2023-02-21 16:55:20.989 [main] INFO com.dtstack.chunjun.util.PluginUtil - ChunJun reset pipeline.jars: [file:/opt/chunjun1.6/chunjun-dist/connector/mysql/chunjun-connector-mysql.jar]
Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: com.dtstack.chunjun.throwable.ChunJunSqlParseException:
----------sql start---------
1>
2>
3> insert into sink select * from source u
----------sql end---------
com/dtstack/chunjun/connector/jdbc/table/JdbcDynamicTableFactory
Unable to create a source for reading table 'default_catalog.default_database.source'.
Table options are:
'connector'='mysql-x'
'password'='******'
'scan.fetch-size'='2'
'scan.query-timeout'='10'
'table-name'='datasource_classify'
'url'='jdbc:mysql://192.168.168.200:3306/iccm_ermp'
'username'='root'
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:366)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:158)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:82)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:117)
at com.dtstack.chunjun.client.util.JobGraphUtil.buildJobGraph(JobGraphUtil.java:62)
at com.dtstack.chunjun.client.standalone.StandaloneClusterClientHelper.submit(StandaloneClusterClientHelper.java:61)
at com.dtstack.chunjun.client.Launcher.main(Launcher.java:119)
Caused by: com.dtstack.chunjun.throwable.ChunJunRuntimeException: com.dtstack.chunjun.throwable.ChunJunSqlParseException:
----------sql start---------
1>
2>
3> insert into sink select * from source u
----------sql end---------
com/dtstack/chunjun/connector/jdbc/table/JdbcDynamicTableFactory
Unable to create a source for reading table 'default_catalog.default_database.source'.
Table options are:
'connector'='mysql-x'
'password'='******'
'scan.fetch-size'='2'
'scan.query-timeout'='10'
'table-name'='datasource_classify'
'url'='jdbc:mysql://192.168.168.200:3306/iccm_ermp'
'username'='root'
at com.dtstack.chunjun.Main.exeSqlJob(Main.java:158)
at com.dtstack.chunjun.Main.main(Main.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349)
... 7 more
Caused by: com.dtstack.chunjun.throwable.ChunJunSqlParseException:
----------sql start---------
1>
2>
3> insert into sink select * from source u
----------sql end---------
com/dtstack/chunjun/connector/jdbc/table/JdbcDynamicTableFactory
Unable to create a source for reading table 'default_catalog.default_database.source'.
Table options are:
'connector'='mysql-x'
'password'='******'
'scan.fetch-size'='2'
'scan.query-timeout'='10'
'table-name'='datasource_classify'
'url'='jdbc:mysql://192.168.168.200:3306/iccm_ermp'
'username'='root'
at com.dtstack.chunjun.sql.parser.SqlParser.lambda$parseSql$1(SqlParser.java:72)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at com.dtstack.chunjun.sql.parser.SqlParser.parseSql(SqlParser.java:66)
at com.dtstack.chunjun.Main.exeSqlJob(Main.java:149)
... 13 more
Caused by: org.apache.flink.table.api.ValidationException: com/dtstack/chunjun/connector/jdbc/table/JdbcDynamicTableFactory
Unable to create a source for reading table 'default_catalog.default_database.source'.
Table options are:
'connector'='mysql-x'
'password'='******'
'scan.fetch-size'='2'
'scan.query-timeout'='10'
'table-name'='datasource_classify'
'url'='jdbc:mysql://192.168.168.200:3306/iccm_ermp'
'username'='root'
at org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:177)
at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.createDynamicTableSource(CatalogSourceTable.java:254)
at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:100)
at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3585)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2507)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2144)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2093)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2050)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:663)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:644)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3438)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:570)
at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:165)
at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:157)
at org.apache.flink.table.planner.operations.SqlToOperationConverter.toQueryOperation(SqlToOperationConverter.java:902)
at org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlQuery(SqlToOperationConverter.java:871)
at org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:250)
at org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlInsert(SqlToOperationConverter.java:564)
at org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:248)
at org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:77)
at org.apache.flink.table.api.internal.StatementSetImpl.addInsertSql(StatementSetImpl.java:50)
at com.dtstack.chunjun.sql.parser.InsertStmtParser.execStmt(InsertStmtParser.java:47)
at com.dtstack.chunjun.sql.parser.AbstractStmtParser.handleStmt(AbstractStmtParser.java:50)
at com.dtstack.chunjun.sql.parser.AbstractStmtParser.handleStmt(AbstractStmtParser.java:52)
at com.dtstack.chunjun.sql.parser.AbstractStmtParser.handleStmt(AbstractStmtParser.java:52)
at com.dtstack.chunjun.sql.parser.SqlParser.lambda$parseSql$1(SqlParser.java:69)
... 24 more
Caused by: java.lang.NoClassDefFoundError: com/dtstack/chunjun/connector/jdbc/table/JdbcDynamicTableFactory
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:471)
at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:282)
at org.apache.flink.table.factories.FactoryUtil.getDynamicTableFactory(FactoryUtil.java:420)
at org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:173)
... 49 more
Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.jdbc.table.JdbcDynamicTableFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 72 more
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Assessment
This issue has not been assessed yet.