caolan / caolan/highland

Working with async streams

Open
#54 8 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

I'm thinking this is mainly a need to document the `end` event somewhere [here](http://highlandjs.org/), but:

Here's a very simple async stream that generates [0,1,2,3,4,5,6,7,8,9,10]:

```
var val = 0;
function generator(push, next) {
process.nextTick(function() {
push(null, val++);
if(val > 10) {
push(null, highland.nil);
}
next();
}
}
var stream = highland(generator);
```

Now I want to write a consumer which counts the number of elements in this stream:

```
var count = 0;
stream
.stopOnError(function(err) {console.log("Bleugh", err.stack);})
.each(function(val) {count++;});
```

Ok... Now, how do I know when it's safe to read the count? How do I know when the stream is done? This seems to work, so I'm guessing this is how I'm supposed to do it:

```
var count = 0;
stream
.on("end", function() {console.log("Count", count);})
.stopOnError(function(err) {console.log("Bleugh", err.stack);})
.each(function(val) {count++;});
```

But since this is undocumented, it seems like something I shouldn't rely on? Is there some method that I missed somewhere? It would be slick if there was a function for this, like say `then()`, and if `each()` returned `this`, then you could:

```
var count = 0;
stream
.each(function(val) {count++;})
.then(function() {console.log("Count", count);})
.stopOnError(function(err) {console.log("Bleugh", err.stack);});
```

Which reads nicely... Although would `then()` get called even if we stopped on an error?

Or am I using your library completely wrong? :P

Contributor guide

Open the contributing guide

Research direction

Start with the async stream example in the issue and review the Highland documentation at highlandjs.org, focusing on the `end` event, `each()`, and `stopOnError()`. Document when the count is safe to read and clarify whether an existing completion method is available, including how completion relates to errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
documentation, stream-processing
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.