MongoEngine / MongoEngine/mongoengine
Reconnect all Replica Set clients after AutoReconnect exception
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
I have several connections registered to the same MongoDb server, but 3 different databases. My web application is run under 4 Gunicorn workers.
I am using a Replica Set.
When the primary is down, the current query fails and a refresh is scheduled in MongoReplicaSetClient (2.8, but I guess in 3.2 it's the same). The next query might succeed if a new primary is selected by the time and MonitorThread gets info about it updating the client's connection.
But the refresh affects only this client. Other clients connected the same MongoDB server are not affected -- the same story happens with each.
This means that if each worker is connected to 3 databases on the same MongoDB server and I am repeating the same HTTP request which uses all 3 databases when a primary fails it takes indefinite time to update all the connected clients. If each HTTP request goes round-robin to each worker out of 4 we need 12 requests to update each Mongo client. But in reality the requests don't go round-robin.
If MongoEngine keeps list of all the connections, I think it's reasonable to disconnect/reconnect all of them (which are connected to the same server) to force connection to the new primary when a connection query fails with AutoReconnect exception.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the MongoReplicaSetClient refresh path and MonitorThread behavior described in the issue, then trace how MongoEngine tracks connections to the same MongoDB server. Done means a failed query with AutoReconnect causes the relevant registered same-server clients to reconnect and select the new primary without requiring separate requests for each client.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100