MongoEngine / MongoEngine/mongoengine

Reconnect all Replica Set clients after AutoReconnect exception

Open
#1,204 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

I have several connections registered to the same MongoDb server, but 3 different databases. My web application is run under 4 Gunicorn workers.
I am using a Replica Set.
When the primary is down, the current query fails and a refresh is scheduled in MongoReplicaSetClient (2.8, but I guess in 3.2 it's the same). The next query might succeed if a new primary is selected by the time and MonitorThread gets info about it updating the client's connection.
But the refresh affects only this client. Other clients connected the same MongoDB server are not affected -- the same story happens with each.
This means that if each worker is connected to 3 databases on the same MongoDB server and I am repeating the same HTTP request which uses all 3 databases when a primary fails it takes indefinite time to update all the connected clients. If each HTTP request goes round-robin to each worker out of 4 we need 12 requests to update each Mongo client. But in reality the requests don't go round-robin.

If MongoEngine keeps list of all the connections, I think it's reasonable to disconnect/reconnect all of them (which are connected to the same server) to force connection to the new primary when a connection query fails with AutoReconnect exception.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the MongoReplicaSetClient refresh path and MonitorThread behavior described in the issue, then trace how MongoEngine tracks connections to the same MongoDB server. Done means a failed query with AutoReconnect causes the relevant registered same-server clients to reconnect and select the new primary without requiring separate requests for each client.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.