MongoEngine / MongoEngine/mongoengine
map_reduce method's behavior can lead to unexpected side effects and it breaks QuerySet chaining
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
Now map_reduce method is messing with you. Why does it change your decisions? Why does it outputs to "inline" collection (it will be explicitely named "inline") when there is ordering on queryset instead of calculating it inline (like expected)?
If it can't sort input documents before doing inline map-reduce it should tell you that and raise an exception or at least raise a warning!
Now it sweeps problem under the carpet and you don't even know that there is any issue. Problem is that it creates some side-effects (new collection) despite it it was told to not do so (inline calculation).
More interesting fact is that you can actually sort input documents even using inline mapReduce:
http://docs.mongodb.org/manual/reference/command/mapReduce/#dbcmd.mapReduce
Moreover it doesn't even sort input documents as it (intuitively) should do. Let see what this code will do:
SomeDocument.objects.filter(key=key).order_by('some_key').map_reduce(map_code, reduce_code)
Some would think that it will make mongo find (filter) documents, sort them (order_by) and finally make map-reduce operation on them. But it actually sorts only output query of map_reduce. Mongoengine's map_reduce docs says that it supports QuerySet chaining but in my opinion this isn't a chain if some operations from front part of chain can actually be executed at the very end.
I think this is a bug or design flaw and should be fixed especially because none of behaviours descibed here are noted in documentation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mongoengine/queryset/base.py around the map_reduce implementation at lines 912 and 924, then compare its behavior with the linked MongoDB mapReduce documentation. Determine how filtered and ordered QuerySets are handled, including inline output and generated collections. Done means the behavior is explicit, ordering is handled or rejected clearly, and the relevant documentation is updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100