MongoEngine / MongoEngine/mongoengine
QuerySetNoCache iterator protocol violation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
I believe that the QuerySetNoCache object violates the iterator protocol because iter is not idempotent. I discovered this with code like the following:
from itertools import *
from mongoengine import *
class Foo(Document):
pass
def chunks(iterator, n):
it = iter(iterator)
while True:
r = tuple(islice(it, n))
if not r:
return
yield r
connect('test')
Foo.objects.delete()
Foo().save()
qs = Foo.objects.no_cache()
for c in chunks(qs, 10):
print c
This is an infinite loop because islice calls iter on its argument, but calling iter on the QuerySetNoCache rewinds the cursor. This can be repaired by changing the body of the __iter__ method to just return self, but that's not backwards compatible as it no longer clones the query set when iter is called on the original object multiple times. Perhaps the solution is something along the lines of
def __iter__(self):
qs = self.clone()
qs.rewind()
while 1:
yield qs.next()
This would fix the issue because the __iter__ method now never returns self, and the generator properly does. On the other hand, this looks odd to me, and I wouldn't submit it in a PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the QuerySetNoCache class and its iter implementation, then reproduce the issue with the chunks and islice example from the report. Done means repeated iteration remains compatible while nested iteration no longer rewinds the cursor or loops indefinitely; add or run focused iterator tests if the repository provides them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100