Automattic / Automattic/monk

Seeming to be a BAD error when using streaming on release 6.0.3

Open
#224 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
1.8k
Forks
180
PR merge metrics
No merged PRs in 30d

Description

I haven't checked for prior releases, but since I haven't seen this before 6.0.3, I suspect this may be tied for the rewrite of monk -- but not sure.

The issue is that code like:

( tasks).find({}, {stream: true})
.each((taskData: TaskData) => {
if (taskData._id) {
mapTasks.set(taskData._id, taskData);
}
})
.catch((err: any) => {
console.error(processErrorMessage(err.toString()));
reject(err);
})
.then(() => {
console.log('tasks #:', mapTasks.size());
resolve(true);
});

doesn't get the full TaskData from the database. It does get all of the items, but ONLY the id's. I was using 'stream: true' because I can have many thousands of items and don't want all of them in memory and the same time.

BUT the code:

( tasks).find({})
.then((docs: TaskData[]) => {
docs.forEach((taskData: TaskData) => {
console.log('listData:', listData)
mapTasks.set(taskData._id, taskData);
});
console.log('docs:', docs.length);
})
.then(() => {
console.log('tasks #:', mapTasks.size());
resolve(true);
});

works properly!

This should be fairly easy to add to your tests, since the mapTasks and all the rest can be ignored.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at Monk's find({}, {stream: true}) entry point and compare its result handling with find({}); reproduce the release 6.0.3 behavior using the TaskData example. Add a regression test covering streamed documents with fields beyond _id, and verify the non-streaming comparison still passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, mongodb
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.