MongoEngine / MongoEngine/mongoengine

Regression on elemMatch?

Open
#1,341 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

Here's a code that works on mongoengine version 0.9.0, but raises exception on 0.10.6:

import mongoengine as mongo


class Top(mongo.Document):
    class Parent(mongo.EmbeddedDocument):
        class Child(mongo.EmbeddedDocument):
            somedata = mongo.IntField()

        value = mongo.DynamicField()  # <-- can be of Child type or untyped (without _cls)

    plist = mongo.ListField(field=mongo.EmbeddedDocumentField(Parent))


def method1():
    # Method 1
    print 'ComboQ:'
    untypedQ = mongo.Q(__raw__={'value': {'_cls': {'$exists': False}}}) & mongo.Q(value__lte=-1)
    childQ = mongo.Q(value=Top.Parent.Child()) & mongo.Q(value__somedata__gte=1)
    comboQ = untypedQ | childQ
    print comboQ.to_query(Top.Parent)

    print 'TopQ:'
    topQ = mongo.Q(plist__match=comboQ.to_query(Top.Parent))
    print topQ.to_query(Top)


def method2():
    # Method 2
    print 'ComboQ:'
    untypedQ = mongo.Q(__raw__={'value': {'_cls': {'$exists': False}, 'lte': -1}})
    childQ = mongo.Q(__raw__={'value': {'_cls': 'Child', 'somedata': {'gte': 1}}})
    comboQ = untypedQ | childQ
    print comboQ.to_query(Top.Parent)

    print 'TopQ:'
    topQ = mongo.Q(plist__match=comboQ.to_query(Top.Parent))
    print topQ.to_query(Top)


if __name__ == '__main__':
    for method in [method1, method2]:
        try:
            method()
        except Exception as e:
            print u'{}: {}'.format(e.__class__.__name__, str(e))
        print

Output on version 0.9.0:

ComboQ:
{'$or': [{'value': {'_cls': {'$exists': False}, '$lte': -1}}, {'value.somedata': {'$gte': 1}, 'value': SON([('_cls', 'Child')])}]}
TopQ:
{'plist': {'$elemMatch': {'$or': [{'value': {'_cls': {'$exists': False}, '$lte': -1}}, {'value.somedata': {'$gte': 1}, 'value': SON([('_cls', 'Child')])}]}}}

ComboQ:
{'$or': [{'value': {'_cls': {'$exists': False}, 'lte': -1}}, {'value': {'somedata': {'gte': 1}, '_cls': 'Child'}}]}
TopQ:
{'plist': {'$elemMatch': {'$or': [{'value': {'_cls': {'$exists': False}, 'lte': -1}}, {'value': {'somedata': {'gte': 1}, '_cls': 'Child'}}]}}}

Output on version 0.10.6:

ComboQ:
{'$or': [{'value': {'_cls': {'$exists': False}, '$lte': -1}}, {'value.somedata': {'$gte': 1}, 'value': SON([('_cls', 'Child')])}]}
TopQ:
InvalidQueryError: Cannot resolve field "$or"

ComboQ:
{'$or': [{'value': {'_cls': {'$exists': False}, 'lte': -1}}, {'value': {'somedata': {'gte': 1}, '_cls': 'Child'}}]}
TopQ:
InvalidQueryError: Cannot resolve field "$or"

Is it regression or some intended design?

Added:
Output on version 0.10.0:

ComboQ:
{'$or': [{'value._cls': {'$exists': False}, 'value': {'$lte': -1}}, {'value.somedata': {'$gte': 1}, 'value': SON([('_cls', 'Child')])}]}
TopQ:
FieldDoesNotExist: The field '$or' does not exist on the document 'Parent'

ComboQ:
{'$or': [{'value._cls': {'$exists': False}, 'lte': -1}, {'value._cls': 'Child', 'somedata': {'gte': 1}}]}
TopQ:
FieldDoesNotExist: The field '$or' does not exist on the document 'Parent'

Which means regression occured between versions 0.10.0 and 0.9.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied reproduction against MongoEngine 0.9.0, 0.10.0, and 0.10.6, focusing on Q.to_query with an $or expression nested in __match. Trace the query transformation that produces the '$or' field-resolution error, and consider the issue done when both methods generate the working elemMatch query shown for 0.9.0 without raising an exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.