MongoEngine / MongoEngine/mongoengine

Saving empty dictionary and list fields throws validation error

Open
#2,320 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

When I save a document with empty list field or dictionary field, mongoengine throws a validation error. It happens either when I set required=True (see Dummy1 below), or if when I use unique_with even though setting required=False and sparse=True (see Dummy2 below). In my documents there are relationships between various fields, dict and list fields can be of variable sizes including 0 elements, and at least one of them is required and their combination must be unique (therefore I use unique_with=True). Currently I could not find a neat solution with mongoengine.

As a workaround validation maybe cancelled in those cases by implementing a custom validating. However, my documents can be very complex. Alternatively, I can define some null dictionary and list, like dict(NULL=True) and ['NULL'] as well as NULL documents for the listfield of reference objects. However, managing these pseudo NULLs for various database operations makes the codes unnecessarily too complicated, while it would be so simple if empty dictionary and lists were allowed.

Perhaps this is a design issue, but I request acceptance of empty dict and list containers (i.e., {} and []). After all, {} and [] are not None, they are valid instances of dict and list.

Or, is it possible not to overwrite the "required" flag when "unique_with" is set, in order to prevent the Validation error in Dummy2 ?

There are related links here:
https://stackoverflow.com/questions/51211880/write-an-empty-dict-field-using-mongoengine
https://github.com/MongoEngine/mongoengine/issues/267

To reproduce try to save Dummy1 and Dummy2 documents:

me.connect('mongoengine_emptydictlist2')

class Dummy1(me.Document):
    val_dict = me.DictField(required=True)
    val_list = me.ListField(required=True)

class Dummy2(me.Document):
    val_str = me.StringField(unique_with=['val_dict', 'val_list'], required=False, sparse=True)
    val_dict = me.DictField(required=False, sparse=True)
    val_list = me.ListField(required=False, sparse=True)

doc1 = Dummy1()
doc2 = Dummy2()

try:
    doc1.save()  # doc1.save(validate=False)
except me.ValidationError as err:
    print(err)

try:
    doc2.save()  # doc2.save(validate=False)
except me.ValidationError as err:
    print(err)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the Dummy1 and Dummy2 reproduction in the issue and record which validation errors occur for empty dictionaries and lists. Trace the relevant field validation and unique_with behavior, then confirm the intended behavior with maintainers; done requires an agreed rule and regression coverage for the reported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.