RocketChat / RocketChat/Rocket.Chat
[BUG] errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 46.1k
- Forks
- 13.9k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 130
Description
Description:
From time to time rocket chat instances show that error:
errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',
But @frdmn advised me that it is an old bug, that was already fixed in #10061 by @sampaiodiego
But our server version is 3.0.3 and we have that bug on a regular basis.
How to fix it? Delete duplicate from mongodb? How to do it correctly?
Expected behavior:
Not have error E11000 duplicate key error index
Actual behavior:
Error and maybe rocket chat instance crash and server resource consume.
Server Setup Information:
- Version of Rocket.Chat Server: 3.0.3
- Operating System: CentOS7
- Deployment Method: docker
- Number of Running Instances: 12
- DB Replicaset Oplog: Enabled
- NodeJS Version: v12.14.0
- MongoDB Version: 4.0.13
Client Setup Information
- Desktop App Version: 2.17.9
- Operating System:
Additional context
Show Read Receipts and Detailed Read Receipts are enabled

Allow Message Editing and Allow Message Deleting are enabled for 10 minutes

Relevant logs:
Logs from Kibana:
March 27th 2020, 12:51:55.623 s-rc-srv-03 rocketchat_rocketchat_3_1 =================================
March 27th 2020, 12:51:55.622 s-rc-srv-03 rocketchat_rocketchat_3_1 Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
March 27th 2020, 12:51:55.622 s-rc-srv-03 rocketchat_rocketchat_3_1 Errors like this can cause oplog processing errors.
March 27th 2020, 12:51:55.622 s-rc-srv-03 rocketchat_rocketchat_3_1 Future node.js versions will automatically exit the process
March 27th 2020, 12:51:55.621 s-rc-srv-03 rocketchat_rocketchat_3_1 [Symbol(mongoErrorContextSymbol)]: {}
March 27th 2020, 12:51:55.621 s-rc-srv-03 rocketchat_rocketchat_3_1 ---------------------------------
March 27th 2020, 12:51:55.621 s-rc-srv-03 rocketchat_rocketchat_3_1 }
March 27th 2020, 12:51:55.620 s-rc-srv-03 rocketchat_rocketchat_3_1 },
March 27th 2020, 12:51:55.620 s-rc-srv-03 rocketchat_rocketchat_3_1 }
March 27th 2020, 12:51:55.615 s-rc-srv-03 rocketchat_rocketchat_3_1 lastOp: [Object]
March 27th 2020, 12:51:55.614 s-rc-srv-03 rocketchat_rocketchat_3_1 nModified: 0,
March 27th 2020, 12:51:55.614 s-rc-srv-03 rocketchat_rocketchat_3_1 nRemoved: 0,
March 27th 2020, 12:51:55.614 s-rc-srv-03 rocketchat_rocketchat_3_1 nMatched: 0,
March 27th 2020, 12:51:55.614 s-rc-srv-03 rocketchat_rocketchat_3_1 upserted: [],
March 27th 2020, 12:51:55.613 s-rc-srv-03 rocketchat_rocketchat_3_1 nUpserted: 0,
March 27th 2020, 12:51:55.613 s-rc-srv-03 rocketchat_rocketchat_3_1 nInserted: 0,
March 27th 2020, 12:51:55.613 s-rc-srv-03 rocketchat_rocketchat_3_1 insertedIds: [Array],
March 27th 2020, 12:51:55.612 s-rc-srv-03 rocketchat_rocketchat_3_1 writeConcernErrors: [],
March 27th 2020, 12:51:55.607 s-rc-srv-03 rocketchat_rocketchat_3_1 writeErrors: [Array],
March 27th 2020, 12:51:55.606 s-rc-srv-03 rocketchat_rocketchat_3_1 result: {
March 27th 2020, 12:51:55.606 s-rc-srv-03 rocketchat_rocketchat_3_1 ok: 1,
March 27th 2020, 12:51:55.605 s-rc-srv-03 rocketchat_rocketchat_3_1 }
March 27th 2020, 12:51:55.605 s-rc-srv-03 rocketchat_rocketchat_3_1 result: BulkWriteResult {
March 27th 2020, 12:51:55.605 s-rc-srv-03 rocketchat_rocketchat_3_1 },
March 27th 2020, 12:51:55.604 s-rc-srv-03 rocketchat_rocketchat_3_1 messageId: 'raufRbopbb9NDcGsQ',
March 27th 2020, 12:51:55.604 s-rc-srv-03 rocketchat_rocketchat_3_1 ts: 2020-03-27T09:51:55.555Z
March 27th 2020, 12:51:55.604 s-rc-srv-03 rocketchat_rocketchat_3_1 userId: 'pE4ztmmZFNJ5TZXFv',
March 27th 2020, 12:51:55.603 s-rc-srv-03 rocketchat_rocketchat_3_1 roomId: '6mkv7tvrJAmi4AJrR',
March 27th 2020, 12:51:55.601 s-rc-srv-03 rocketchat_rocketchat_3_1 _id: 'DyBPDuqasguFt2BCD',
March 27th 2020, 12:51:55.597 s-rc-srv-03 rocketchat_rocketchat_3_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',
March 27th 2020, 12:51:55.597 s-rc-srv-03 rocketchat_rocketchat_3_1 op: {
March 27th 2020, 12:51:55.596 s-rc-srv-03 rocketchat_rocketchat_3_1 err: {
March 27th 2020, 12:51:55.596 s-rc-srv-03 rocketchat_rocketchat_3_1 index: 0,
March 27th 2020, 12:51:55.596 s-rc-srv-03 rocketchat_rocketchat_3_1 code: 11000,
March 27th 2020, 12:51:55.595 s-rc-srv-03 rocketchat_rocketchat_3_1 name: 'BulkWriteError',
March 27th 2020, 12:51:55.595 s-rc-srv-03 rocketchat_rocketchat_3_1 driver: true,
March 27th 2020, 12:51:55.595 s-rc-srv-03 rocketchat_rocketchat_3_1 },
March 27th 2020, 12:51:55.594 s-rc-srv-03 rocketchat_rocketchat_3_1 ts: 2020-03-27T09:51:55.555Z
March 27th 2020, 12:51:55.592 s-rc-srv-03 rocketchat_rocketchat_3_1 messageId: 'raufRbopbb9NDcGsQ',
March 27th 2020, 12:51:55.590 s-rc-srv-03 rocketchat_rocketchat_3_1 userId: 'pE4ztmmZFNJ5TZXFv',
March 27th 2020, 12:51:55.589 s-rc-srv-03 rocketchat_rocketchat_3_1 _id: 'DyBPDuqasguFt2BCD',
March 27th 2020, 12:51:55.589 s-rc-srv-03 rocketchat_rocketchat_3_1 roomId: '6mkv7tvrJAmi4AJrR',
March 27th 2020, 12:51:55.588 s-rc-srv-03 rocketchat_rocketchat_3_1 code: 11000,
March 27th 2020, 12:51:55.588 s-rc-srv-03 rocketchat_rocketchat_3_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',
March 27th 2020, 12:51:55.588 s-rc-srv-03 rocketchat_rocketchat_3_1 op: {
March 27th 2020, 12:51:55.587 s-rc-srv-03 rocketchat_rocketchat_3_1 at processTicksAndRejections (internal/process/task_queues.js:75:11) {
March 27th 2020, 12:51:55.587 s-rc-srv-03 rocketchat_rocketchat_3_1 index: 0,
March 27th 2020, 12:51:55.587 s-rc-srv-03 rocketchat_rocketchat_3_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:51:55.586 s-rc-srv-03 rocketchat_rocketchat_3_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:51:55.581 s-rc-srv-03 rocketchat_rocketchat_3_1 at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
March 27th 2020, 12:51:55.580 s-rc-srv-03 rocketchat_rocketchat_3_1 BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }
March 27th 2020, 12:51:55.580 s-rc-srv-03 rocketchat_rocketchat_3_1 at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
March 27th 2020, 12:51:55.566 s-rc-srv-03 rocketchat_rocketchat_3_1 === UnHandledPromiseRejection ===
March 27th 2020, 12:39:26.826 s-rc-srv-01 rocketchat_rocketchat_2_1 Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
March 27th 2020, 12:39:26.826 s-rc-srv-01 rocketchat_rocketchat_2_1 =================================
March 27th 2020, 12:39:26.826 s-rc-srv-01 rocketchat_rocketchat_2_1 Future node.js versions will automatically exit the process
March 27th 2020, 12:39:26.822 s-rc-srv-01 rocketchat_rocketchat_2_1 Errors like this can cause oplog processing errors.
March 27th 2020, 12:39:26.820 s-rc-srv-01 rocketchat_rocketchat_2_1 [Symbol(mongoErrorContextSymbol)]: {}
March 27th 2020, 12:39:26.820 s-rc-srv-01 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:39:26.820 s-rc-srv-01 rocketchat_rocketchat_2_1 ---------------------------------
March 27th 2020, 12:39:26.819 s-rc-srv-01 rocketchat_rocketchat_2_1 lastOp: [Object]
March 27th 2020, 12:39:26.819 s-rc-srv-01 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:39:26.819 s-rc-srv-01 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:39:26.819 s-rc-srv-01 rocketchat_rocketchat_2_1 upserted: [],
March 27th 2020, 12:39:26.818 s-rc-srv-01 rocketchat_rocketchat_2_1 nRemoved: 0,
March 27th 2020, 12:39:26.818 s-rc-srv-01 rocketchat_rocketchat_2_1 nMatched: 0,
March 27th 2020, 12:39:26.818 s-rc-srv-01 rocketchat_rocketchat_2_1 nModified: 0,
March 27th 2020, 12:39:26.815 s-rc-srv-01 rocketchat_rocketchat_2_1 nUpserted: 0,
March 27th 2020, 12:39:26.814 s-rc-srv-01 rocketchat_rocketchat_2_1 writeConcernErrors: [],
March 27th 2020, 12:39:26.814 s-rc-srv-01 rocketchat_rocketchat_2_1 insertedIds: [Array],
March 27th 2020, 12:39:26.814 s-rc-srv-01 rocketchat_rocketchat_2_1 nInserted: 0,
March 27th 2020, 12:39:26.814 s-rc-srv-01 rocketchat_rocketchat_2_1 writeErrors: [Array],
March 27th 2020, 12:39:26.813 s-rc-srv-01 rocketchat_rocketchat_2_1 result: BulkWriteResult {
March 27th 2020, 12:39:26.813 s-rc-srv-01 rocketchat_rocketchat_2_1 ok: 1,
March 27th 2020, 12:39:26.813 s-rc-srv-01 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:39:26.813 s-rc-srv-01 rocketchat_rocketchat_2_1 result: {
March 27th 2020, 12:39:26.806 s-rc-srv-01 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:39:26.804 s-rc-srv-01 rocketchat_rocketchat_2_1 roomId: 'GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN',
March 27th 2020, 12:39:26.804 s-rc-srv-01 rocketchat_rocketchat_2_1 userId: 'GMqJTNEx74S3oGGLZ',
March 27th 2020, 12:39:26.804 s-rc-srv-01 rocketchat_rocketchat_2_1 messageId: 'gu9BrisadJMaEcFRn',
March 27th 2020, 12:39:26.804 s-rc-srv-01 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:39:26.770Z
March 27th 2020, 12:39:26.803 s-rc-srv-01 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:39:26.803 s-rc-srv-01 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:39:26.803 s-rc-srv-01 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }',
March 27th 2020, 12:39:26.803 s-rc-srv-01 rocketchat_rocketchat_2_1 _id: 'o9W4gbNQCnWvPa23R',
March 27th 2020, 12:39:26.802 s-rc-srv-01 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:39:26.802 s-rc-srv-01 rocketchat_rocketchat_2_1 err: {
March 27th 2020, 12:39:26.798 s-rc-srv-01 rocketchat_rocketchat_2_1 driver: true,
March 27th 2020, 12:39:26.797 s-rc-srv-01 rocketchat_rocketchat_2_1 name: 'BulkWriteError',
March 27th 2020, 12:39:26.796 s-rc-srv-01 rocketchat_rocketchat_2_1 userId: 'GMqJTNEx74S3oGGLZ',
March 27th 2020, 12:39:26.796 s-rc-srv-01 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:39:26.796 s-rc-srv-01 rocketchat_rocketchat_2_1 messageId: 'gu9BrisadJMaEcFRn',
March 27th 2020, 12:39:26.796 s-rc-srv-01 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:39:26.770Z
March 27th 2020, 12:39:26.795 s-rc-srv-01 rocketchat_rocketchat_2_1 _id: 'o9W4gbNQCnWvPa23R',
March 27th 2020, 12:39:26.795 s-rc-srv-01 rocketchat_rocketchat_2_1 roomId: 'GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN',
March 27th 2020, 12:39:26.795 s-rc-srv-01 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:39:26.794 s-rc-srv-01 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:39:26.794 s-rc-srv-01 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:39:26.794 s-rc-srv-01 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }',
March 27th 2020, 12:39:26.791 s-rc-srv-01 rocketchat_rocketchat_2_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:39:26.791 s-rc-srv-01 rocketchat_rocketchat_2_1 at processTicksAndRejections (internal/process/task_queues.js:75:11) {
March 27th 2020, 12:39:26.790 s-rc-srv-01 rocketchat_rocketchat_2_1 at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
March 27th 2020, 12:39:26.790 s-rc-srv-01 rocketchat_rocketchat_2_1 at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
March 27th 2020, 12:39:26.790 s-rc-srv-01 rocketchat_rocketchat_2_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:39:26.789 s-rc-srv-01 rocketchat_rocketchat_2_1 BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }
March 27th 2020, 12:39:26.777 s-rc-srv-01 rocketchat_rocketchat_2_1 === UnHandledPromiseRejection ===
March 27th 2020, 12:36:13.659 s-rc-srv-02 rocketchat_rocketchat_2_1 Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
March 27th 2020, 12:36:13.659 s-rc-srv-02 rocketchat_rocketchat_2_1 =================================
March 27th 2020, 12:36:13.659 s-rc-srv-02 rocketchat_rocketchat_2_1 Errors like this can cause oplog processing errors.
March 27th 2020, 12:36:13.659 s-rc-srv-02 rocketchat_rocketchat_2_1 Future node.js versions will automatically exit the process
March 27th 2020, 12:36:13.658 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:36:13.658 s-rc-srv-02 rocketchat_rocketchat_2_1 ---------------------------------
March 27th 2020, 12:36:13.658 s-rc-srv-02 rocketchat_rocketchat_2_1 [Symbol(mongoErrorContextSymbol)]: {}
March 27th 2020, 12:36:13.657 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:36:13.655 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:36:13.654 s-rc-srv-02 rocketchat_rocketchat_2_1 lastOp: [Object]
March 27th 2020, 12:36:13.653 s-rc-srv-02 rocketchat_rocketchat_2_1 nModified: 0,
March 27th 2020, 12:36:13.653 s-rc-srv-02 rocketchat_rocketchat_2_1 upserted: [],
March 27th 2020, 12:36:13.653 s-rc-srv-02 rocketchat_rocketchat_2_1 nRemoved: 0,
March 27th 2020, 12:36:13.652 s-rc-srv-02 rocketchat_rocketchat_2_1 nMatched: 0,
March 27th 2020, 12:36:13.652 s-rc-srv-02 rocketchat_rocketchat_2_1 nInserted: 0,
March 27th 2020, 12:36:13.652 s-rc-srv-02 rocketchat_rocketchat_2_1 nUpserted: 0,
March 27th 2020, 12:36:13.651 s-rc-srv-02 rocketchat_rocketchat_2_1 writeConcernErrors: [],
March 27th 2020, 12:36:13.651 s-rc-srv-02 rocketchat_rocketchat_2_1 insertedIds: [Array],
March 27th 2020, 12:36:13.650 s-rc-srv-02 rocketchat_rocketchat_2_1 writeErrors: [Array],
March 27th 2020, 12:36:13.646 s-rc-srv-02 rocketchat_rocketchat_2_1 ok: 1,
March 27th 2020, 12:36:13.645 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:36:13.645 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:36:13.645 s-rc-srv-02 rocketchat_rocketchat_2_1 result: BulkWriteResult {
March 27th 2020, 12:36:13.645 s-rc-srv-02 rocketchat_rocketchat_2_1 result: {
March 27th 2020, 12:36:13.644 s-rc-srv-02 rocketchat_rocketchat_2_1 userId: 'QFNqm28joYJqhmBCA',
March 27th 2020, 12:36:13.644 s-rc-srv-02 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:36:13.620Z
March 27th 2020, 12:36:13.644 s-rc-srv-02 rocketchat_rocketchat_2_1 _id: 'tz4GPFEmF2NmwHpzY',
March 27th 2020, 12:36:13.644 s-rc-srv-02 rocketchat_rocketchat_2_1 roomId: 'xnGcER23hDhCaXS6G',
March 27th 2020, 12:36:13.644 s-rc-srv-02 rocketchat_rocketchat_2_1 messageId: 'Ewi50RwMhZLGEJfv6',
March 27th 2020, 12:36:13.643 s-rc-srv-02 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:36:13.641 s-rc-srv-02 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:36:13.641 s-rc-srv-02 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:36:13.641 s-rc-srv-02 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }',
March 27th 2020, 12:36:13.640 s-rc-srv-02 rocketchat_rocketchat_2_1 name: 'BulkWriteError',
March 27th 2020, 12:36:13.640 s-rc-srv-02 rocketchat_rocketchat_2_1 driver: true,
March 27th 2020, 12:36:13.640 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:36:13.640 s-rc-srv-02 rocketchat_rocketchat_2_1 err: {
March 27th 2020, 12:36:13.639 s-rc-srv-02 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:36:13.620Z
March 27th 2020, 12:36:13.639 s-rc-srv-02 rocketchat_rocketchat_2_1 userId: 'QFNqm28joYJqhmBCA',
March 27th 2020, 12:36:13.639 s-rc-srv-02 rocketchat_rocketchat_2_1 messageId: 'Ewi50RwMhZLGEJfv6',
March 27th 2020, 12:36:13.638 s-rc-srv-02 rocketchat_rocketchat_2_1 roomId: 'xnGcER23hDhCaXS6G',
March 27th 2020, 12:36:13.634 s-rc-srv-02 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:36:13.634 s-rc-srv-02 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:36:13.634 s-rc-srv-02 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }',
March 27th 2020, 12:36:13.634 s-rc-srv-02 rocketchat_rocketchat_2_1 _id: 'tz4GPFEmF2NmwHpzY',
March 27th 2020, 12:36:13.633 s-rc-srv-02 rocketchat_rocketchat_2_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:36:13.633 s-rc-srv-02 rocketchat_rocketchat_2_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:36:13.633 s-rc-srv-02 rocketchat_rocketchat_2_1 at processTicksAndRejections (internal/process/task_queues.js:75:11) {
March 27th 2020, 12:36:13.633 s-rc-srv-02 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:36:13.632 s-rc-srv-02 rocketchat_rocketchat_2_1 at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
March 27th 2020, 12:36:13.632 s-rc-srv-02 rocketchat_rocketchat_2_1 at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
March 27th 2020, 12:36:13.627 s-rc-srv-02 rocketchat_rocketchat_2_1 BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }
March 27th 2020, 12:36:13.626 s-rc-srv-02 rocketchat_rocketchat_2_1 === UnHandledPromiseRejection ===
March 27th 2020, 12:34:14.506 s-rc-srv-02 rocketchat_rocketchat_2_1 =================================
March 27th 2020, 12:34:14.505 s-rc-srv-02 rocketchat_rocketchat_2_1 Future node.js versions will automatically exit the process
March 27th 2020, 12:34:14.503 s-rc-srv-02 rocketchat_rocketchat_2_1 Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
March 27th 2020, 12:34:14.502 s-rc-srv-02 rocketchat_rocketchat_2_1 Errors like this can cause oplog processing errors.
March 27th 2020, 12:34:14.502 s-rc-srv-02 rocketchat_rocketchat_2_1 ---------------------------------
March 27th 2020, 12:34:14.501 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:34:14.501 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:34:14.501 s-rc-srv-02 rocketchat_rocketchat_2_1 [Symbol(mongoErrorContextSymbol)]: {}
March 27th 2020, 12:34:14.501 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:34:14.500 s-rc-srv-02 rocketchat_rocketchat_2_1 nRemoved: 0,
March 27th 2020, 12:34:14.500 s-rc-srv-02 rocketchat_rocketchat_2_1 lastOp: [Object]
March 27th 2020, 12:34:14.500 s-rc-srv-02 rocketchat_rocketchat_2_1 upserted: [],
March 27th 2020, 12:34:14.498 s-rc-srv-02 rocketchat_rocketchat_2_1 nModified: 0,
March 27th 2020, 12:34:14.495 s-rc-srv-02 rocketchat_rocketchat_2_1 nMatched: 0,
March 27th 2020, 12:34:14.495 s-rc-srv-02 rocketchat_rocketchat_2_1 nInserted: 0,
March 27th 2020, 12:34:14.495 s-rc-srv-02 rocketchat_rocketchat_2_1 nUpserted: 0,
March 27th 2020, 12:34:14.494 s-rc-srv-02 rocketchat_rocketchat_2_1 writeErrors: [Array],
March 27th 2020, 12:34:14.494 s-rc-srv-02 rocketchat_rocketchat_2_1 writeConcernErrors: [],
March 27th 2020, 12:34:14.494 s-rc-srv-02 rocketchat_rocketchat_2_1 ok: 1,
March 27th 2020, 12:34:14.494 s-rc-srv-02 rocketchat_rocketchat_2_1 insertedIds: [Array],
March 27th 2020, 12:34:14.493 s-rc-srv-02 rocketchat_rocketchat_2_1 result: {
March 27th 2020, 12:34:14.493 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:34:14.493 s-rc-srv-02 rocketchat_rocketchat_2_1 result: BulkWriteResult {
March 27th 2020, 12:34:14.492 s-rc-srv-02 rocketchat_rocketchat_2_1 }
March 27th 2020, 12:34:14.490 s-rc-srv-02 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:34:14.457Z
March 27th 2020, 12:34:14.489 s-rc-srv-02 rocketchat_rocketchat_2_1 roomId: 'A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5',
March 27th 2020, 12:34:14.489 s-rc-srv-02 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:34:14.489 s-rc-srv-02 rocketchat_rocketchat_2_1 userId: 'A9ag6QqqTqYnKDnaf',
March 27th 2020, 12:34:14.489 s-rc-srv-02 rocketchat_rocketchat_2_1 messageId: 'fSgY8fg4BL9zWTBxi',
March 27th 2020, 12:34:14.489 s-rc-srv-02 rocketchat_rocketchat_2_1 _id: 'FAL7JejWpQcvCHKoK',
March 27th 2020, 12:34:14.488 s-rc-srv-02 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:34:14.488 s-rc-srv-02 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }',
March 27th 2020, 12:34:14.488 s-rc-srv-02 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:34:14.487 s-rc-srv-02 rocketchat_rocketchat_2_1 err: {
March 27th 2020, 12:34:14.486 s-rc-srv-02 rocketchat_rocketchat_2_1 driver: true,
March 27th 2020, 12:34:14.484 s-rc-srv-02 rocketchat_rocketchat_2_1 ts: 2020-03-27T09:34:14.457Z
March 27th 2020, 12:34:14.484 s-rc-srv-02 rocketchat_rocketchat_2_1 },
March 27th 2020, 12:34:14.484 s-rc-srv-02 rocketchat_rocketchat_2_1 name: 'BulkWriteError',
March 27th 2020, 12:34:14.484 s-rc-srv-02 rocketchat_rocketchat_2_1 messageId: 'fSgY8fg4BL9zWTBxi',
March 27th 2020, 12:34:14.483 s-rc-srv-02 rocketchat_rocketchat_2_1 op: {
March 27th 2020, 12:34:14.483 s-rc-srv-02 rocketchat_rocketchat_2_1 _id: 'FAL7JejWpQcvCHKoK',
March 27th 2020, 12:34:14.483 s-rc-srv-02 rocketchat_rocketchat_2_1 roomId: 'A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5',
March 27th 2020, 12:34:14.483 s-rc-srv-02 rocketchat_rocketchat_2_1 userId: 'A9ag6QqqTqYnKDnaf',
March 27th 2020, 12:34:14.482 s-rc-srv-02 rocketchat_rocketchat_2_1 code: 11000,
March 27th 2020, 12:34:14.482 s-rc-srv-02 rocketchat_rocketchat_2_1 errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }',
March 27th 2020, 12:34:14.481 s-rc-srv-02 rocketchat_rocketchat_2_1 index: 0,
March 27th 2020, 12:34:14.478 s-rc-srv-02 rocketchat_rocketchat_2_1 at processTicksAndRejections (internal/process/task_queues.js:75:11) {
March 27th 2020, 12:34:14.478 s-rc-srv-02 rocketchat_rocketchat_2_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:34:14.477 s-rc-srv-02 rocketchat_rocketchat_2_1 at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
March 27th 2020, 12:34:14.477 s-rc-srv-02 rocketchat_rocketchat_2_1 at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
March 27th 2020, 12:34:14.477 s-rc-srv-02 rocketchat_rocketchat_2_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:34:14.476 s-rc-srv-02 rocketchat_rocketchat_2_1 BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }
March 27th 2020, 12:34:14.465 s-rc-srv-02 rocketchat_rocketchat_2_1 === UnHandledPromiseRejection ===
March 27th 2020, 12:31:17.852 s-rc-srv-01 rocketchat_rocketchat_1_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:31:17.852 s-rc-srv-01 rocketchat_rocketchat_1_1 at processTicksAndRejections (internal/process/task_queues.js:75:11)
March 27th 2020, 12:31:17.852 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:31:17.849 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:1259:5
March 27th 2020, 12:31:17.849 s-rc-srv-01 rocketchat_rocketchat_1_1 at handleCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:128:55)
March 27th 2020, 12:31:17.848 s-rc-srv-01 rocketchat_rocketchat_1_1 at executeCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:395:17)
March 27th 2020, 12:31:17.848 s-rc-srv-01 rocketchat_rocketchat_1_1 at deleteCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:349:17)
March 27th 2020, 12:31:17.848 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:364:54
March 27th 2020, 12:31:17.848 s-rc-srv-01 rocketchat_rocketchat_1_1 at ClientSession.endSession (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/sessions.js:134:41)
March 27th 2020, 12:31:17.847 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/gridfs-stream/index.js:188:25
March 27th 2020, 12:31:17.847 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:410:17
March 27th 2020, 12:31:17.847 s-rc-srv-01 rocketchat_rocketchat_1_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:398:11
March 27th 2020, 12:31:17.846 s-rc-srv-01 rocketchat_rocketchat_1_1 Error: FileNotFound: no file with id JwNf9f6ePkM7tKM8M found
March 27th 2020, 12:13:18.506 s-rc-srv-02 rocketchat_rocketchat_4_1 at processTicksAndRejections (internal/process/task_queues.js:75:11)
March 27th 2020, 12:13:18.506 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:1259:5
March 27th 2020, 12:13:18.506 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
March 27th 2020, 12:13:18.506 s-rc-srv-02 rocketchat_rocketchat_4_1 at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
March 27th 2020, 12:13:18.505 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:364:54
March 27th 2020, 12:13:18.505 s-rc-srv-02 rocketchat_rocketchat_4_1 at handleCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:128:55)
March 27th 2020, 12:13:18.503 s-rc-srv-02 rocketchat_rocketchat_4_1 at deleteCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:349:17)
March 27th 2020, 12:13:18.503 s-rc-srv-02 rocketchat_rocketchat_4_1 at executeCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:395:17)
March 27th 2020, 12:13:18.499 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/gridfs-stream/index.js:188:25
March 27th 2020, 12:13:18.499 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:410:17
March 27th 2020, 12:13:18.499 s-rc-srv-02 rocketchat_rocketchat_4_1 at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:398:11
March 27th 2020, 12:13:18.499 s-rc-srv-02 rocketchat_rocketchat_4_1 at ClientSession.endSession (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/sessions.js:134:41)
March 27th 2020, 12:13:18.498 s-rc-srv-02 rocketchat_rocketchat_4_1 Error: FileNotFound: no file with id HC2JgAwfeCAyfGEZ6 found`
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the read-receipt writes to the rocketchat_message_read_receipt collection and the prior fix referenced in issue #10061. Reproduce the duplicate-key error with the reported MongoDB setup, then verify that repeated receipt processing no longer produces E11000 errors or unhandled promise rejections.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, typescript
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100