MongoEngine / MongoEngine/mongoengine
Cannot save documents with dots in a field key.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
Issue https://github.com/MongoEngine/mongoengine/issues/1594 (related: https://github.com/MongoEngine/mongoengine/pull/2193) suggests that it has resolved the issue with having dots in a key name. However, this does't seem to be the case. Tested with mongoengine==0.22.1 and pymongo==3.11.2.
Although I'm raising it here I think that this is most likely a limitation of pymongo itself as https://docs.mongodb.com/manual/core/document/#field-names suggests that dots in key names aren't supported by official MongoDB drivers. However, I was just hoping to get a sanity check that https://github.com/MongoEngine/mongoengine/issues/1594 / https://github.com/MongoEngine/mongoengine/pull/2193 don't actually resolve the problem. I feel like in fields#DictField.validate that key_has_dot_or_dollar should be added back in (or possibly just a modified key_has_dot helper function)
For confirmation, I've tested with a $ in the key name and this works - it's just . that doesn't work.
A simple test case
import mongoengine
from mongoengine import fields, Document
mongoengine.connect(db="my_test_db")
class DocumentWithDots(Document):
data = fields.DictField()
if __name__ == '__main__':
DocumentWithDots(data={"t.est": "data"}).save()
Expected
document to be saved without issue
Actual
Traceback (most recent call last):
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/mongodot.py", line 14, in <module>
DocumentWithDots(data={"t.est": "data"}).save()
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/mongoengine/document.py", line 398, in save
object_id = self._save_create(doc, force_insert, write_concern)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/mongoengine/document.py", line 463, in _save_create
object_id = wc_collection.insert_one(doc).inserted_id
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 701, in insert_one
session=session),
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 615, in _insert
bypass_doc_val, session)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 603, in _insert_one
acknowledged, _insert_command, session)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1498, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1416, in _retry_internal
return func(session, sock_info, retryable)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 598, in _insert_command
retryable_write=retryable_write)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/pool.py", line 699, in command
self._raise_connection_failure(error)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/pool.py", line 694, in command
exhaust_allowed=exhaust_allowed)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/network.py", line 122, in command
codec_options, ctx=compression_ctx)
File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/message.py", line 715, in _op_msg
flags, command, identifier, docs, check_keys, opts)
bson.errors.InvalidDocument: key 't.est' must not contain '.'
Process finished with exit code 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fields#DictField.validate and the key_has_dot_or_dollar or key_has_dot helper mentioned in the report. Reproduce the failure using the provided DocumentWithDots example and check how MongoEngine and PyMongo handle the dotted key. Done means the supported behavior is confirmed and covered by an appropriate regression test or documented limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100