caolan / caolan/highland

Concatenated Node 0.8 streams don't get proper backpressure

Open
#87 19 comments 0 reactions 0 assignees View on GitHub
bug interop
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

If you use `_.concat` to concatenate two event-driven Node streams (e.g. a pair of `gulp.src()` streams), you can lose items unless you explicitly `.pipe()` the streams to new Highland Streams. That is, the following will not always output everything matched by both patterns:

```
_.concat(gulp.src(pattern1),
gulp.src(pattern2))
.toArray(console.log)
```

But this will:

```
_.concat(gulp.src(pattern1).pipe(_()),
gulp.src(pattern2).pipe(_()))
.toArray(console.log)
```

The problem appears to be caused by Highland's lazy initialization of converted Node streams, causing some items to be dropped. In particular, the stream being concatenated isn't listened to until the entire first stream has been consumed. (Hence, the workaround of explicitly piping.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.