Uniqueness within ListFields

Feature Request

Description

Uniqueness within ListFields would allow MongoEngine to assert uniqueness of lists containing fields and embedded documents, within the same document.

Background

Currently MongoDB does not enforce unique indices within the same document. For example:

class EmbDoc(EmbeddedDocument):
    x = StringField(required=True, unique=True)

class Doc(Document):
    l = ListField(EmbeddedDocumentField(EmbDoc))

d = Doc(l=[EmbDoc(x='t'), EmbDoc(x='t')]).save()  # No Exception

The saved document will contain duplicate entries for x, despite it being marked as unique. This is to be expected and is the normal behavior of MongoDB as of today.

Looking at http://docs.mongodb.org/manual/core/index-unique/ we can see the following:

The unique constraint applies to separate documents in the collection. That is, the unique index prevents separate documents from having the same value for the indexed key, but the index does not prevent a document from having multiple elements or embedded documents in an indexed array from having the same value.

More information about the issue can be found at http://joegornick.com/2012/10/25/mongodb-unique-indexes-on-single-embedded-documents/

MongoDB has had an open issue to add support for this since 2010. https://jira.mongodb.org/browse/SERVER-1068

Proposed Solution

I propose that until MongoDB addresses the above linked issue, if they ever do, uniqueness with ListFields is asserted via MongoEngine implementation.

If MongoDB is to ever address the issue, the internal implementation for MongoEngine can be switched to use Unique 'Doc' Indices in the same manner that unique fields are handled today via unique collection indices.

Solution Conceptualization 1

Add a doc_unique Boolean parameter to Fields .

class EmbDoc(EmbeddedDocument):
    x = StringField(required=True, unique=True, doc_unique=True)

class Doc(Document):
    l = ListField(EmbeddedDocumentField(EmbDoc))
   s = ListField(StringField(), doc_unique=True)

d = Doc(l=[EmbDoc(x='t'), EmbDoc(x='t')]).save()  # Now raises an exception.
d = Doc(s=['Conflict', 'Conflict']).save() # Now raises an exception.

Pros:

• Simple to understand.
• No syntax or identifier to parse.
• Contained within the Embedded Document (the parent doesn't need to know the doc_unique field).

Cons:

• Parameter would serve no purpose unless the field is inside of a ListField, which can lead to user confusion.

---

Solution Conceptualization 2

Add a doc_unique String parameter to ListField. The Id would be a string containing a '$' wildcard character used to represent an index in the list.

class EmbDoc(EmbeddedDocument):
    x = StringField(required=True, unique=True)

class Doc(Document):
    l = ListField(EmbeddedDocumentField(EmbDoc), doc_unique='$.x')
    s = ListField(StringField(), doc_unique='$')

d = Doc(l=[EmbDoc(x='t'), EmbDoc(x='t')]).save()  # Now raises an exception.
d = Doc(s=['Conflict', 'Conflict']).save() # Now raises an exception.

Pros:

• Only need to add a new parameter to a single field type.
• Can only be used when it is allowed to be used.

Cons:

• Parsing an identifier.
• Human readability and user usability.
• Parent must be made aware of any Embedded Document fields that need to be document unique.

Start with the existing unique-field handling and ListField/EmbeddedDocument behavior described in the issue, then compare the two proposed doc_unique designs against MongoDB's SERVER-1068 limitation. Done requires choosing and implementing a design that rejects duplicate values within a document while preserving current uniqueness behavior.