Concatenated Node 0.8 streams don't get proper backpressure
- Dominant language
- JavaScript
- Stars
- 3.4k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
If you use `_.concat` to concatenate two event-driven Node streams (e.g. a pair of `gulp.src()` streams), you can lose items unless you explicitly `.pipe()` the streams to new Highland Streams. That is, the following will not always output everything matched by both patterns:
```
_.concat(gulp.src(pattern1),
gulp.src(pattern2))
.toArray(console.log)
```
But this will:
```
_.concat(gulp.src(pattern1).pipe(_()),
gulp.src(pattern2).pipe(_()))
.toArray(console.log)
```
The problem appears to be caused by Highland's lazy initialization of converted Node streams, causing some items to be dropped. In particular, the stream being concatenated isn't listened to until the entire first stream has been consumed. (Hence, the workaround of explicitly piping.)
Contributor guide
Assessment
This issue has not been assessed yet.