MongoEngine / MongoEngine/mongoengine

Transaction fails when multiple database aliases use different MongoDB clients

Open
#2,908 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

When configuring multiple database connections with different host parameters, an InvalidOperation error is raised when executing a transaction that spans multiple models bound to different database aliases.

class DocDefault(Document):
    id = StringField(primary_key=True)

    meta = {
        'db_alias': 'default'
    }


class DocAdmin(Document):
    id = StringField(primary_key=True)

    meta = {
        'db_alias': 'admin'
    }

if __name__ == '__main__':
    # This configuration works correctly
    right_config = [
        {
            'db': 'default',
            'alias': 'default'
        },
        {
            'db': 'admin',
            'alias': 'admin'
        },
    ]

    # This configuration triggers the error
    wrong_config = [
        {
            'db': 'default',
            'alias': 'default',
            'host': 'mongodb://127.0.0.1:27017/default',
        },
        {
            'db': 'admin',
            'alias': 'admin',
            'host': 'mongodb://127.0.0.1:27017/admin',
        },
    ]

    for c in wrong_config:
        connect(**c)

    with run_in_transaction():
        doc_default = DocDefault(id='a')
        doc_default.save()

        doc_admin = DocAdmin(id='a')
        doc_admin.save()
        print(doc_default)
        print(doc_admin)

The following exception is raised:

pymongo.errors.InvalidOperation: Can only use session with the MongoClient that started it

With wrong_config, two distinct MongoClient instances are created because the host parameters differ.
The transaction context currently manages a single session stack and does not differentiate between database aliases.
Consequently, models bound to the admin alias incorrectly inherit the session belonging to the default alias's client, leading to the InvalidOperation error.
With right_config, the same MongoClient is reused (since only db and alias are provided), so the error does not occur.

Would it make sense to consider managing transactions on a per-database-alias basis? I'm not entirely sure if this is the most appropriate solution, but perhaps differentiating sessions by alias could help ensure that the correct MongoClient and session are used for each operation. Curious to hear if there might be a better approach or if I'm misunderstanding the intended behavior. Thanks for taking a look!

(My English is not very good, so I used AI assistance for this translation. I apologize for any mistakes or awkward phrasing.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the transaction context behind run_in_transaction() and the connection setup used by connect(); inspect how sessions are stored and selected for database aliases. Reproduce the wrong_config case with the two Document classes, then verify that each alias uses the MongoClient that created its session and that the cross-alias transaction no longer raises InvalidOperation.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.