ReferenceFields defaulted to None in a MapField fail validation and turns into wrong DBRefs

So I created an Article document, where one of the fields would be a map of references to translated articles, keyed by the language code. For sake of this error report, I simplified it down to and reproduced with below document definition:

class TestArticle(Document):
    translations_i18n = MapField(ReferenceField("TestArticle"), default={"en": None, "sv": None})
    test_ref = ReferenceField("TestArticle", default=None)

If I create a blank new TestArticle and save, it will throw a Validation error:

ta = TestArticle()
ta.save()
---
mongoengine.errors.ValidationError: ValidationError (TestArticle:None) (en.A ReferenceField only accepts DBRef, LazyReference, ObjectId or documents sv.A ReferenceField only accepts DBRef, LazyReference, ObjectId or documents: ['translations_i18n'])

However, if I comment out translations_i18n, it doesn't throw an error when setting just the single ReferenceField to None. So this is a bit unexpected.

Now, in my case, I already had documents before I added translations_i18n. And when I get them from database using Mongoengine, and it goes through dereferencing, it converts the None above to newly generated ObjectIds, pointing nowhere in the database. So if I fetched a document, that had no translations_i18n value in the database, it turned into a dict with dummy DBRefs after de-referencing.

ta = TestArticle.objects(id="5f6861747373807535abb743").first()
check = ta.translations_i18n
print(check)
---
{'sv': DBRef('test_article', ObjectId('5f685ba25e78cf20bc2062c3')), 'en': DBRef('test_article', ObjectId('5f685c0a5e78cf20bc2062c4'))}

The reason None turns into dummy references seems to be ObjectIdField.to_python(self, value) which generates new ObjectIds from None. I haven't been able to fully understand the flow, but apparently None values for the ReferenceField called test_ref doesn't get to this point.

This was tricky behaviour for me to debug. I realize it may not be expected to set a MapField with ReferenceFields to a default None, and I have worked around this in my app. But it still seems like unexpected behaviour worthy of an error message and a test case. What's your take?

Start with the MapField and ReferenceField validation flow, then trace ObjectIdField.to_python(self, value) for None using the reproduction in the issue. Compare the MapField behavior with the standalone test_ref case; done should include a regression test and the agreed validation or dereferencing behavior, with the requested error message if that is the chosen outcome.