hdfsreader: fastjson error in the latest version
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
2023-10-09 17:48:09.632 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"hdfsreader",
"parameter":{
"column":[
"*"
],
"defaultFS":"hdfs://10.130.1.6:9000",
"encoding":"UTF-8",
"fieldDelimiter":"\t",
"fileType":"text",
"path":"/user/datax/student.txt"
}
},
"writer":{
"name":"mysqlwriter",
"parameter":{
"column":[
"id",
"name"
],
"connection":[
{
"jdbcUrl":"jdbc:mysql://10.130.1.6:3306/datax?useSSL=false",
"table":[
"student2"
]
}
],
"password":"******",
"username":"root",
"writeMode":"insert"
}
}
}
],
"setting":{
"speed":{
"channel":"1"
}
}
}
2023-10-09 17:48:09.654 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false
2023-10-09 17:48:09.655 [main] INFO JobContainer - DataX jobContainer starts job.
2023-10-09 17:48:09.656 [main] INFO JobContainer - Set jobId = 0
2023-10-09 17:48:09.669 [job-0] INFO HdfsReader$Job - init() begin...
2023-10-09 17:48:10.117 [job-0] ERROR JobContainer - Exception when job run
java.lang.StackOverflowError: null
at java.base/java.lang.ClassLoader.definePackage(ClassLoader.java:2106) ~[na:na]
at java.base/java.lang.ClassLoader.lambda$packages$4(ClassLoader.java:2372) ~[na:na]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na]
at java.base/java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3603) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[na:na]
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517) ~[na:na]
at java.base/java.lang.ClassLoader.getDefinedPackages(ClassLoader.java:2271) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFuncFinal.getFieldValue(FieldWriterObjectFuncFinal.java:26) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:52) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:86) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.ObjectWriter5.write(ObjectWriter5.java:90) ~[fastjson2-2.0.23.jar:na]
at com.alibaba.fastjson2.writer.OWG_4_8_Module.write(Unknown Source) ~[na:na]
at com.alibaba.fastjson2.writer.FieldWriterObjectFinal.write(FieldWriterObjectFinal.java:81) ~[fastjson2-2.0.23.jar:na]
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the HdfsReader-to-MySQL job with the JSON configuration and Java stack trace shown in the issue, then inspect the fastjson2 serialization path involved during HdfsReader initialization. Done means the job initializes without the reported StackOverflowError; the issue does not name repository files or tests to run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100