alibaba / alibaba/DataX

datax 更新数据报错

Open
#837 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

"upsertInfo": {
"isUpsert": "true",
"upsertKey": "_id"
}

datax 用这种方式更新数据 报如下 错误 :

Caused by: com.mongodb.MongoBulkWriteException: Bulk write operation error on server 192.168.1.144:3717. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error collection: nfplus.userInfo_bak index: _id_ dup key: { : "cd849555c29a11e7b0e77cd30abdd3e8" }', details={ }}].
at com.mongodb.connection.BulkWriteBatchCombiner.getError(BulkWriteBatchCombiner.java:176)
at com.mongodb.connection.BulkWriteBatchCombiner.throwOnError(BulkWriteBatchCombiner.java:205)
at com.mongodb.connection.BulkWriteBatchCombiner.getResult(BulkWriteBatchCombiner.java:146)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:190)
at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:168)
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:230)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:221)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:168)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:74)
at com.mongodb.Mongo.execute(Mongo.java:781)
at com.mongodb.Mongo$2.execute(Mongo.java:764)
at com.mongodb.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:323)
at com.mongodb.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:311)
at com.alibaba.datax.plugin.writer.mongodbwriter.MongoDBWriter$Task.doBatchInsert(MongoDBWriter.java:312)
at com.alibaba.datax.plugin.writer.mongodbwriter.MongoDBWriter$Task.startWrite(MongoDBWriter.java:138)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
at java.lang.Thread.run(Thread.java:748)

![image](https://user-images.githubusercontent.com/7835704/93981038-fe8e5180-fdb1-11ea-9c72-53912b7d26d6.png)

如何 解决?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the MongoDBWriter.java paths shown in the stack trace, especially doBatchInsert at line 312 and startWrite at line 138, and compare them with the provided upsertInfo configuration. Reproduce the duplicate-key case against MongoDB; done means the configured update no longer fails with the reported E11000 error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mongodb
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.