Automattic / Automattic/mongoose
Add `Query.prototype.findAndCount()` for composable paginated queries
- Dominant language
- JavaScript
- Stars
- 27.5k
- Forks
- 4k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 35
Description
### Prerequisites
- [x] I have written a descriptive issue title
- [x] I have searched existing issues to ensure the feature has not already been requested
### 🚀 Feature Proposal
Hi @vkarpov15, thank you for considering the `Model.findAndCount()` proposal I opened in [#16454](https://github.com/Automattic/mongoose/issues/16454), and for implementing it in Mongoose `9.10.0` through [#16460](https://github.com/Automattic/mongoose/pull/16460).
It would also be useful to expose the same operation on an existing `Query` instance, especially when queries are built incrementally and passed between functions.
Would you consider adding `Query.prototype.findAndCount()` so that a previously constructed find query can be executed as follows?
```javascript
const query = buildQuery(model, filters);
const [documents, total] = await query
.skip(40)
.limit(20)
.findAndCount();
```
The proposed method would be a terminal operation returning a promise that resolves to `[documents, total]`, consistent with `Model.findAndCount()`.
### Motivation
Reusable query-building functions can apply filters and sorting, then return a query for another function to paginate and execute.
Adopting `Model.findAndCount()` in this situation means extracting the existing query's filter, projection and options and constructing another model-level call. A method on `Query` would allow the caller to execute the query it already has.
The following example illustrates this pattern using a generic model and placeholder fields.
### Example
### Build a query
Here, `model` is any Mongoose model with `status` and `createdAt` fields. This synchronous helper returns an unexecuted query.
```javascript
function buildQuery(model, filters = {}) {
const query = model.find();
if (filters.status) {
query.where('status').equals(filters.status);
}
query.sort({ createdAt: -1, _id: -1 });
return query;
}
```
### Current approach: clone the query for data and count
```javascript
async function getPage(model, filters, offset, limit) {
const query = buildQuery(model, filters);
const dataQuery = query.clone();
const countQuery = query.clone();
return {
documents: await dataQuery.skip(offset).limit(limit),
total: await countQuery.countDocuments(),
};
}
```
The count must use the same filtering conditions without inheriting the pagination applied to the data query. This requires manually coordinating the two queries at each call site or writing a custom wrapper.
### Proposed approach: execute findAndCount() on the existing query
The query builder above would remain unchanged. Only the execution step would change:
```javascript
async function getPage(model, filters, offset, limit) {
const query = buildQuery(model, filters);
const [documents, total] = await query
.skip(offset)
.limit(limit)
.findAndCount();
return { documents, total };
}
```
For example, if 135 documents match `{ status: 'active' }`, an offset of 40 and a limit of 20 would return up to 20 documents in the configured order, with `total === 135`. A page beyond the last result would return an empty array while retaining the same total.
## Expected behavior
- Use the conditions and applicable options already configured on the query, including collation, hints and supported execution options.
- Apply sorting, pagination, projection, populate and lean settings to the returned documents as a normal find query would.
- Count all documents matching the filter, independently of `skip` and `limit`.
- Run the normal `find` and `countDocuments` middleware.
- Preserve TypeScript inference for the returned documents, including lean and populated result types where supported.
- Keep the execution and consistency semantics of `Model.findAndCount()`, including its requirements for `sort` and `limit`. The example supplies sorting in the query builder and pagination in the calling function.
The main benefit would be being able to adopt `findAndCount()` directly in code that already composes and passes around Mongoose queries, while keeping existing query builders intact.
Contributor guide
Research direction
Start by reading the existing Model.findAndCount() implementation and the Query.prototype execution and countDocuments paths. Trace how query options, pagination, middleware, populate, lean, and TypeScript types are handled, then verify that Query.prototype.findAndCount() returns paginated documents with an independently filtered total and preserves the documented semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, mongodb, node.js
- Domain
- api, backend, database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100