MongoEngine / MongoEngine/mongoengine

Problem handling dots in DictField keys

Open
#2,539 5 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

Consider the following code:

class TestModel(db.Document):
    test = db.DictField()

obj = TestModel()
obj.test['key.with.dot'] = 'abc'
obj.save()

I think this should work, because MongoDB allows dots in dict key names as of version 3.6.
But in fact I get the following error:

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    obj.save()
  File "mongoengine/document.py", line 450, in save
    self._clear_changed_fields()
  File "mongoengine/base/document.py", line 547, in _clear_changed_fields
    field_name = data._reverse_db_field_map.get(part, part)
AttributeError: 'NoneType' object has no attribute '_reverse_db_field_map'

After some digging I found that _clear_changed_fields takes list of string keys from _get_changed_fields() where each key is a dot-delimited path into the object. Obviously this doesn't play well with keys containing dots.

I think it would be safer to store keys as tuples rather than dot-delimited strings, but it will probably involve too many changes?

Tested with MongoEngine==0.23.1 and python 3.9.5.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided TestModel reproduction and trace _get_changed_fields() into _clear_changed_fields() in mongoengine/base/document.py. Check how dot-delimited paths and DictField keys are represented, then verify that saving a key containing a dot completes without the reported AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.