TransformerRegistry加载自定义插件的问题

你好 我在使用datax 23-9月版本时发现一个bug
在json里设置splitPk时,他会分割n个线程都会去对自定义的插件去进行反射初始化,会导致同一个类会多次加载然后报错
个人建议是在
com.alibaba.datax.core.transport.transformer.TransformerRegistry.loadTransformer(String each)
方法里 Class transformerClass = jarLoader.loadClass(className);后加一行
if (registedTransformer.containsKey(funName)) {
return;
}
如果被加载过了,因为分割导致的其他线程不需要去再去加载了,不然
registComplexTransformer->
if (registedTransformer.containsKey(complexTransformer.getTransformerName())) {
会抛出已经被加载过的问题
同时registedTransformer在使用splitPk时 建议使用更安全的ConcurrentHashMap

Start with com.alibaba.datax.core.transport.transformer.TransformerRegistry.loadTransformer(String each), then trace registComplexTransformer and the registedTransformer map while splitPk creates multiple threads. Confirm the duplicate custom-plugin loading path and define done as preventing repeated registration or loading safely under concurrent use without the existing already-loaded error.

Written by the indexing model from the issue text.