MongoEngine / MongoEngine/mongoengine

EmbeddedDocumentField errors during update when adding new field to DynamicEmbeddedDocument

Open
#2,486 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

Summary: When using a DynamicEmbeddedDocument, an InvalidQueryError occurs if you try to set a field that is not defined in the MongoEngine schema. This invalidates the purpose of a dynamic document.

Example:

class B(DynamicEmbeddedDocument):
   foo = StringField()

class A(Document):
  b = EmbeddedDocumentField(B)

# This works
A.objects(...).update(set__b__foo="123")

# This does not work
A.objects(...).update(set__b__bar="123")
# mongoengine.errors.InvalidQueryError: Cannot resolve field "bar"

Solution:
In my custom fork of MongoEngine, I've updated the EmbeddedDocumentField to account for dynamic documents as such.

class EmbeddedDocumentField(BaseField):
      def lookup_member(self, member_name):
        doc_and_subclasses = [self.document_type] + self.document_type.__subclasses__()
        for doc_type in doc_and_subclasses:
            field = doc_type._fields.get(member_name)
            if field:
                return field
        # NEW CODE
        if member_name not in ("S", "$") and any(issubclass(doc_type, DynamicEmbeddedDocument) for doc_type in doc_and_subclasses):
           return DynamicField(db_field=member_name)

Custom Situation: I encountered this error because I had a EmbeddedDocumentListField of DynamicEmbeddedDocuments and I was trying to perform array operators like so:

A.objects(b__foo="abc").modify(set__list_of_b__S__bar="123")

If the maintainers find my solution acceptable, I would be happy to write tests and make a PR with it. Otherwise, I will defer to coders with greater expertise in this codebase for the proper strategy here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the provided update examples and inspect the EmbeddedDocumentField.lookup_member implementation. Add regression tests for setting an undefined field on a DynamicEmbeddedDocument, including the EmbeddedDocumentListField array-operator case, and consider the S and $ path markers shown in the proposed change. Done means these updates no longer raise InvalidQueryError while normal field lookup remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.