Inconsistent behavior of auth options in host string and kwarg when connect

Background

MongoDB version: 4.2.6
mongoengine version: 0.23.1
pymongo: 3.12.0

In my scenario, I need to connect with authentication_source and authentication_mechanism and then do some queries.

My code goes like this,

class User(Document):
    name = StringField()

# connect
connect(
    host='mongodb://{username}:{password}@{host}:{port}/{db}'.format(**MONGO_CONFIG),
    authentication_mechanism=MONGO_CONFIG['mechanism'],
    authentication_source=MONGO_CONFIG['db']
)

# query
User.objects.count()

Analysis

1. If auth options to be a kwarg of connect

authentication_mechanism and authentication_source will be cleaned when _create_connection,

https://github.com/MongoEngine/mongoengine/blob/5fe9436ea7bd2f47812b017104285210848a1276/mongoengine/connection.py#L265-L275

And will get MongoCredential in pymongo like this when connect,

MongoCredential(mechanism='DEFAULT', source='s', username='u', password='p', mechanism_properties=None, cache=<pymongo.auth._Cache object at 0x7f9aa2db0e50>)

The error will be raised when we do a query or a count op, because the new MongoCredential is different

Traceback (most recent call last):
  File "debug.py", line 27, in <module>
    User.objects.count()
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/mongoengine/queryset/manager.py", line 38, in __get__
    queryset = queryset_class(owner, owner._get_collection())
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/mongoengine/document.py", line 215, in _get_collection
    db = cls._get_db()
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/mongoengine/document.py", line 193, in _get_db
    return get_db(cls._meta.get("db_alias", DEFAULT_CONNECTION_NAME))
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/mongoengine/connection.py", line 368, in get_db
    conn_settings["username"], conn_settings["password"], **auth_kwargs
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/pymongo/database.py", line 1575, in authenticate
    connect=True)
  File "/Users/.../opt/anaconda3/envs/mongoengine-pr/lib/python3.7/site-packages/pymongo/mongo_client.py", line 806, in _cache_credentials
    raise OperationFailure('Another user is already authenticated '
pymongo.errors.OperationFailure: Another user is already authenticated to this database. You must logout first.

MongoCredential(mechanism='SCRAM-SHA-256', source='s', username='u', password='p', mechanism_properties=None, cache=<pymongo.auth._Cache object at 0x7f9fe769b390>)

authentication_mechanism

The root cause of this issue is that the get_db function is handling the transform (mongoengine's authentication_mechanism to pymongo's mechanism) when a query is taking off. However, that is not happening during the connection phase.

https://github.com/MongoEngine/mongoengine/blob/5fe9436ea7bd2f47812b017104285210848a1276/mongoengine/connection.py#L356

authentication_source

authentication_source is not handling during the connection phase too. But this will be set as db name by default in pymongo.

2. If auth options to be in the host string

Everything goes fine in this situation because pymongo will parse the host string and set them correctly. Like,

connect(host='mongodb://{username}:{password}@{host}:{port}/{db}?authMechanism={mechanism}&authSource={db}'.format(**MONGO_CONFIG))

https://github.com/mongodb/mongo-python-driver/blob/2eb0df812c6a4afbdbcd12692ca8da0b4dd0c14e/pymongo/mongo_client.py#L659

So should we reconcile them?

Start in mongoengine/connection.py, comparing _create_connection with get_db, and then review PyMongo's MongoClient option parsing at the linked entry point. Reproduce the shown connection and User.objects.count() flow with options in the host string and as kwargs; done means both forms produce consistent authentication behavior without the credential conflict.