MongoEngine / MongoEngine/mongoengine

Slicing a ListField(EmbeddedField) can lead to strange value in _changed_fields

Open
#1,855 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug
Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

While fixing something else in BaseList, I noticed that slicing a BaseList could lead to bad values of the parent document._changed_fields. Additionally, it is behaving differently under python2 and python3.

In fact, see below (under PY3):

class Age(me.EmbeddedDocument):
    age = me.IntField()

class Person(me.Document):
    ages = me.EmbeddedDocumentListField(Age)

p = Person()
p.ages = [Age(age=1), Age(age=2), Age(age=3)]
p.save()

sliced = p.ages[1:3]
print(sliced)			# [<Age: Age object>, <Age: Age object>]
print(type(sliced))		# <class 'mongoengine.base.datastructures.BaseList'>
print(sliced._instance) 	# Person object
print(sliced._name)		# "ages.slice(1, 3, None)" <---- BOOM!
print(p._changed_fields)	# []
sliced[1] = Age(age=1000)
print(p._changed_fields)	# ['ages.slice(1, 3, None).1'] >
p.save()			# Fails with OperationError: Could not save document (cannot use the part (ages of ages.slice(1, 3, None).1)...

Under PY2, here is the output:

class Age(me.EmbeddedDocument):
    age = me.IntField()

class Person(me.Document):
    ages = me.EmbeddedDocumentListField(Age)


p = Person()
p.ages = [Age(age=1), Age(age=2), Age(age=3)]
p.save()

sliced = p.ages[1:3]
print(sliced)			# [<Age: Age object>, <Age: Age object>]
print(type(sliced))		# list (and not a BaseList like in PY3)
print(sliced._instance)		# Fails with AttributeError: 'list' object has no attribute '_instance'

The difference between PY2 & PY3 comes from the fact that PY2 calls BaseList.getslice and PY3 calls getitem (with key being a slice, which causes the weird ._name attribute). This is something I'm harmonizing in another PR (#1853) so that at least it will behave the same (but faulty).

Being able to track changes properly after slicing is not trivial, especially if we try to identify specifically the index/key of the things being changed. I'll think of it in the next days, if I manage to find a clean fix, I'll push a PR

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in BaseList, comparing its getslice and getitem handling for slices, and review the behavior described for Python 2 and Python 3. Check PR #1853 for the related harmonization, then define tests that cover slicing, _changed_fields, and saving the parent document without the reported OperationError.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.