Embedded collections doesn't work with gorm 7.3.0
- Dominant language
- Groovy
- Stars
- 2.9k
- Forks
- 975
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 92
Description
Hi @puneetbehl !
In our system we have a large number of domain classes. In our recent upgrade to grails 5 (5.3.2) that makes use of GORM 7.3.0 of our system, which was working fine with the Grails 2.5.6, we noticed that the embedded entities were not loaded in the object that contained it: For example,
```groovy
class Country {
List regions
static embedded = ['regions']
}
```
In the above `Country` has embedded `regions`, these regions are initialized as an `empty list` when loaded through GORM. Whereas, in the database collection `Country` has `regions`. Even, when using the MongoDB API, these regions are loaded correctly as list of `Document`.
This does not work when using either `Country.findAll()`, or `Country.createCriteria().list{}`. It maybe breaking with other GORM queries as well.
Upon, debugging, we noticed that the problem might be caused because the database returns a `Map` instead of a `Document`. The Groovy class `MongoEntityPersister` exclusively checks for the class `Document` which breaks certain conditions. For example:
1. `MongoEntityPersister.loadEmbeddedCollection()`.
When the method is executed, I noticed that in our case it goes through the `else` branch because the following condition evaluates to `false`
```
(if(Map.class.isAssignableFrom(embeddedCollection.getType())))
```
The above results in embedded `regions` to return an empty `List` because it is a `Map`.
```
if (embeddedInstances instanceof Collection) { // true
Collection coll = (Collection)embeddedInstances;
for (Object dbo : coll) { // for each embedded element...
if (dbo instanceof Document) { // <------ this condition is problematic: is a Map, not a Document
Document nativeEntry = (Document)dbo;
```
By modifying the above to following, might fixes the problem:
```
if (embeddedInstances instanceof Collection) {
Collection coll = (Collection)embeddedInstances;
for (Object dbo : coll) {
if (dbo instanceof Map) { // Map, instead Document, in this section
Document nativeEntry;
if(dbo instanceof Document){ // then, it is a Document? then the same thing
nativeEntry = (Document)dbo;
}
else{
nativeEntry = new Document((Map)dbo); // if it is a Map, a new Document instance is created based on that Map.
}
```
The above change solves the our problem where the `regions` are correctly loaded via the GORM `Country.findAll` method.
Doing more tests I notice that there is a second problem: if this embedded entity uses composition, some related attributes are loaded and others are not.
For example: my country has regions, and a region has a region type. This type of region is a Domain. The regions are embedded in the country, and a region has only one type of region, as an associated instance (composition).
This instance, the region type, is not loaded.
Performing more debugging, I find that the code goes by:
NativeEntityPersister#refreshObjectStateFromNativeEntry(PersistentEntity, Object, Serializable, T, boolean)
for (final PersistentProperty prop : props) { // for each attribute of my region, in the case of regionType...
... more code ...
else if (prop instanceof ToOne) { // true
if (prop instanceof Embedded) { // true
Embedded embedded = (Embedded) prop; // isnt null, ok
if(embedded.getAssociatedEntity() != null) { // true
T embeddedEntry = getEmbedded(nativeEntry, propKey); // <----- THIS section returns null. Why?
... continuing debugging leads to this method ...
public abstract class AbstractMongoObectEntityPersister extends NativeEntryEntityPersister { ...
... more code ...
protected T getEmbedded(T nativeEntry, String key) {
Object embeddedDocument = this.getValueRetrievalStrategy().getValue(nativeEntry, key);
return this.isEmbeddedEntry(embeddedDocument) ? embeddedDocument : null; // <-------- THIS section is the problem...
}
... continuing debugging leads to this method ...
In MongoEntityPersister....
@Override
protected boolean isEmbeddedEntry(Object entry) {
return entry instanceof Document;
}
...clearly this method gives true when it is a Document, but my region type appears as Map, which gives false. This, by giving false, the caller (AbstractMongoObectEntityPersister#getEmbedded(T, String)) determines that it is null, so it does not load my region type.
Making this change should fix it:
@Override
protected boolean isEmbeddedEntry(Object entry) {
return entry instanceof Map;
}
The problem now is that internally we get various errors related to the Map and Document type. And these are related to the getValue() and setValue() methods of the MongoEntityPersister class:
public static final ValueRetrievalStrategy VALUE_RETRIEVAL_STRATEGY = new ValueRetrievalStrategy() {
@Override
public Object getValue(Document document, String name) { // <---- Designed for Document, but internally from mongo you get a Map
return document.get(name);
}
@Override
public void setValue(Document document, String name, Object value) { // <---- Designed for Document, but internally from mongo you get a Map
document.put(name, value);
}
};
This is the error you get after making all the changes I mentioned above:
```
Caused by: java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class org.bson.Document (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; org.bson.Document is in unnamed module of loader 'app')
at org.grails.datastore.mapping.mongo.engine.MongoEntityPersister$1.getValue(MongoEntityPersister.java:58)
at org.grails.datastore.mapping.mongo.engine.AbstractMongoObectEntityPersister.discriminatePersistentEntity(AbstractMongoObectEntityPersister.java:315)
at org.grails.datastore.mapping.engine.NativeEntryEntityPersister.createObjectFromEmbeddedNativeEntry(NativeEntryEntityPersister.java:368)
at org.grails.datastore.mapping.engine.NativeEntryEntityPersister.refreshObjectStateFromNativeEntry(NativeEntryEntityPersister.java:443)
at org.grails.datastore.mapping.mongo.engine.AbstractMongoObectEntityPersister.refreshObjectStateFromNativeEntry(AbstractMongoObectEntityPersister.java:278)
at org.grails.datastore.mapping.engine.NativeEntryEntityPersister.createObjectFromEmbeddedNativeEntry(NativeEntryEntityPersister.java:370)
at org.grails.datastore.mapping.mongo.engine.MongoEntityPersister.loadEmbeddedCollection(MongoEntityPersister.java:115)
at org.grails.datastore.mapping.engine.NativeEntryEntityPersister.refreshObjectStateFromNativeEntry(NativeEntryEntityPersister.java:507)
at org.grails.datastore.mapping.mongo.engine.AbstractMongoObectEntityPersister.refreshObjectStateFromNativeEntry(AbstractMongoObectEntityPersister.java:281)
at org.grails.datastore.mapping.engine.NativeEntryEntityPersister.createObjectFromNativeEntry(NativeEntryEntityPersister.java:363)
at org.grails.datastore.mapping.mongo.query.MongoQuery$MongoResultList.convertDBObject(MongoQuery.java:1392)
at org.grails.datastore.mapping.mongo.query.MongoQuery$MongoResultList.convertObject(MongoQuery.java:1378)
at org.grails.datastore.gorm.query.AbstractResultList.convertObject(AbstractResultList.java:99)
at org.grails.datastore.gorm.query.AbstractResultList.initializeFully(AbstractResultList.java:59)
at org.grails.datastore.gorm.query.AbstractResultList.size(AbstractResultList.java:206)
at grails.gorm.PagedResultList.size(PagedResultList.java:114)
at grails.gorm.PagedResultList.isEmpty(PagedResultList.java:119)
at org.grails.plugins.web.rest.render.html.DefaultHtmlRenderer.resolveModelVariableName(DefaultHtmlRenderer.groovy:105)
at org.grails.plugins.web.rest.render.html.DefaultHtmlRenderer.applyModel(DefaultHtmlRenderer.groovy:88)
at org.grails.plugins.web.rest.render.html.DefaultHtmlRenderer.render(DefaultHtmlRenderer.groovy:79)
at grails.artefact.controller.RestResponder$Trait$Helper.internalRespond(RestResponder.groovy:216)
at grails.artefact.controller.RestResponder$Trait$Helper.respond(RestResponder.groovy:83)
at
... 72 common frames omitted
```
As a solution we will try to use an older version of gorm, but if this problem can be solved it would be great since it is always necessary to have the latest. If you need more information, please comment and I will respond as soon as possible.
Some details:
| Grails Version: 5.3.2
| JVM Version: 11.0.16.1
| Gorm version: 7.3.0
| Windows 11, with last updates.
(build.gradle)
implementation 'org.grails.plugins:mongodb:7.3.0'
implementation "org.grails:gorm-mongodb-spring-boot:7.3.0"
configurations.all {
// Genera conflictos, tomaba el 3.6 viejo.
exclude module:'mongodb-driver'
}
Contributor guide
Research direction
Start with MongoEntityPersister.loadEmbeddedCollection() and isEmbeddedEntry(), then trace the calls through AbstractMongoObectEntityPersister#getEmbedded and NativeEntityPersister#refreshObjectStateFromNativeEntry. Reproduce using the Country/regions example and the reported Grails 5.3.2 and GORM 7.3.0 setup; done means embedded collections and their associated region types load correctly through GORM queries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100