Automattic / Automattic/mongoose

Optimize nested populate

Open
#3,812 3 comments 6 reactions 0 assignees View on GitHub
performance
Dominant language
JavaScript
Stars
27.5k
Forks
4k
Avg merge
2d 7h
Merged PRs (30d)
35

Description

In the following `PostSchema` we have nested structure that has `_author` field in post itself and comments:

```
var PostSchema = mongoose.Schema({
_author: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
comments: {
type: [{
_author: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},

text: {
type: String
}
}]
}
});

var PostModel = mongoose.model('Post', PostSchema);
```

When we fetch post and populate `_author` fields mongoose makes two queries:

```
PostModel.findById(ObjectId(...)).populate('_author comments._author').exec(callback);

Mongoose: users.find({ _id: { '$in': [ ObjectId("1") ] } }) { fields: undefined }
Mongoose: users.find({ _id: { '$in': [ ObjectId("2") ] } }) { fields: undefined }
```

It would be great performance increment if mongoose will make only one query to populate `_author` field:

```
Mongoose: users.find({ _id: { '$in': [ ObjectId("1"), ObjectId("2")] } }) { fields: undefined }
```

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by tracing how populate handles the top-level _author and comments._author paths, then verify the query logging for the issue's PostSchema example. Done means both references to User are batched into one users.find query without changing populated results.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, mongodb, node.js
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.