caolan / caolan/highland

back-propagation of _.nil

Open
#172 43 comments 0 reactions 0 assignees View on GitHub
3.x
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

Hello,

I have been trying the following scenario :

```
var resource = open()
var s = _(function(push, next) {
..use resource to push elements and _.nil when resource is empty ..
})
.consume(function(err, x, push, next) {
if (err === null) {
push(err);
next();
}
else if (x === _.nil) {
resource.close();
push(null, x);
}
else {
push(null,x);
}
})
```

I open a resource, consume from it, and wait for _.nil to close the resource.

This works well if I consume all the tokens with

```
s.resume();
```

now if i do

```
s.take(10).resume();
```

The resource is never closed, because take seem to send _.nil downstream but not inform upstream that they will not be pulled from again.

You might say that I could close the resource by catching the _.nil after take(10) but this is not what I want to do because I do not want the downstream code to have a reference to the resource.

Would it make sense to have back-propagation of _.nil or a mechanism to inform upstream streams that they will not be pulled from again ?

In node.js streams, when you do

```
s1.pipe(s2)
```

when s2 sends a 'close' event, s1 automatically unpipes s2 for example and you have a way to detect that nothing will be pulled again.

https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L568

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the difference between s.resume() and s.take(10).resume(), focusing on _.nil propagation and the upstream behavior described for take. Compare this with the referenced Node.js readable-stream close handling, then establish the desired lifecycle semantics and regression coverage before implementing a solution.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.