MongoEngine / MongoEngine/mongoengine
Slicing a ListField(EmbeddedField) can lead to strange value in _changed_fields
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
While fixing something else in BaseList, I noticed that slicing a BaseList could lead to bad values of the parent document._changed_fields. Additionally, it is behaving differently under python2 and python3.
In fact, see below (under PY3):
class Age(me.EmbeddedDocument):
age = me.IntField()
class Person(me.Document):
ages = me.EmbeddedDocumentListField(Age)
p = Person()
p.ages = [Age(age=1), Age(age=2), Age(age=3)]
p.save()
sliced = p.ages[1:3]
print(sliced) # [<Age: Age object>, <Age: Age object>]
print(type(sliced)) # <class 'mongoengine.base.datastructures.BaseList'>
print(sliced._instance) # Person object
print(sliced._name) # "ages.slice(1, 3, None)" <---- BOOM!
print(p._changed_fields) # []
sliced[1] = Age(age=1000)
print(p._changed_fields) # ['ages.slice(1, 3, None).1'] >
p.save() # Fails with OperationError: Could not save document (cannot use the part (ages of ages.slice(1, 3, None).1)...
Under PY2, here is the output:
class Age(me.EmbeddedDocument):
age = me.IntField()
class Person(me.Document):
ages = me.EmbeddedDocumentListField(Age)
p = Person()
p.ages = [Age(age=1), Age(age=2), Age(age=3)]
p.save()
sliced = p.ages[1:3]
print(sliced) # [<Age: Age object>, <Age: Age object>]
print(type(sliced)) # list (and not a BaseList like in PY3)
print(sliced._instance) # Fails with AttributeError: 'list' object has no attribute '_instance'
The difference between PY2 & PY3 comes from the fact that PY2 calls BaseList.getslice and PY3 calls getitem (with key being a slice, which causes the weird ._name attribute). This is something I'm harmonizing in another PR (#1853) so that at least it will behave the same (but faulty).
Being able to track changes properly after slicing is not trivial, especially if we try to identify specifically the index/key of the things being changed. I'll think of it in the next days, if I manage to find a clean fix, I'll push a PR
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in BaseList, comparing its getslice and getitem handling for slices, and review the behavior described for Python 2 and Python 3. Check PR #1853 for the related harmonization, then define tests that cover slicing, _changed_fields, and saving the parent document without the reported OperationError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100