caolan / caolan/highland

How to terminate a generator should be documented

Open
#696 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

The docs say this about generators:

> **Generators** - These are functions which provide values for the Stream. They are lazy and can be infinite, they can also be asynchronous (for example, making a HTTP request). You emit values on the Stream by calling push(err, val), much like a standard Node.js callback. Once it has been called, the generator function will not be called again unless you call next(). This call to next() will signal you've finished processing the current data and allow for the generator function to be called again. If the Stream is still being consumed the generator function will then be called again.
> You can also redirect a generator Stream by passing a new source Stream to read from to next. For example: next(other_stream) - then any subsequent calls will be made to the new source.

I implemented the described interface. My generator is paging through HTTP responses. Once the pages were exhausted however the final consumption functions (like `collect()` or `toArray()` or `toPromise()`) were not being called.

I struggled for some time, thinking the problem was with my stream pipeline somewhere.

Only later did I discover that the stream source itself was still open waiting for a call to `next()`. By looking at other issues in this repo I eventually discovered the `highland.nil` "end of stream marker" Adding `push(null, highland.nil)` to my generator once the pages were exhausted instantly closed the stream and my problem was resolved.

This may seem obvious to those familiar with highland, but it was totally a surprise to me. Also I realize `highland.nil` clearly states it is the end of stream marker, but I hadn't browsed that part of the docs. Ideally I think the generator docs should mention something about terminating itself.

Finally.. before I added `highland.nil` my generator actually had no dependency on highland itself, which was quite nice. Now however it does. Is there a way to terminate the generator without requiring highland in my generator module?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.