alibaba / alibaba/DataX

MongoDB中的文档类型转换失败

Open
#101 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

报如下错误:
com.mongodb.util.JSONParseException:
Document{{name=张, age=25}}
^
at com.mongodb.util.JSONParser.parse(JSON.java:230) ~[mongo-java-driver-3.2.2.jar:na]
at com.mongodb.util.JSONParser.parse(JSON.java:155) ~[mongo-java-driver-3.2.2.jar:na]
at com.mongodb.util.JSON.parse(JSON.java:92) ~[mongo-java-driver-3.2.2.jar:na]
at com.mongodb.util.JSON.parse(JSON.java:73) ~[mongo-java-driver-3.2.2.jar:na]
at com.alibaba.datax.plugin.writer.mongodbwriter.MongoDBWriter$Task.doBatchInsert(MongoDBWriter.java:232) [mongodbwriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.mongodbwriter.MongoDBWriter$Task.startWrite(MongoDBWriter.java:144) [mongodbwriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) [datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]

[问题原因]
mongodbreader使用Document对文档类型进行存储,而mongodbwriter使用BasicBSONObject对象来进行存储,Document对象的toString()方法会输出类似Document{}的结构,导致mongodbwriter使用JSON.parse解析时失败。应统一存储格式为BSONObject

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with MongoDBWriter.java at the batch-insert path around lines 144 and 232, then compare how mongodbreader stores document values. Verify the failing Document{{name=张, age=25}} case and trace where its string form reaches JSON.parse. Done means reader and writer use a consistent BSON-compatible representation without the reported JSONParseException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mongodb
Domain
data-engineering, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.