caolan / caolan/highland

End stream within map()

Open
#513 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
3.4k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

How can I force a stream to end from within a map() function?

Scenario:
- Processing a very large stream of data
- Calling a promise via `.map($(promise)).merge()`
- The promise called above is to an external API that has a hard QUERY LIMIT.
- When the query limit is detected (via value on the stream === QUERY_LIMIT_MESSAGE), I would like to terminate all processing.

Contributor guide

Open the contributing guide

Research direction

No repository file or test is named. Start by reading the stream API documentation and the entry points for map() and merge(), then trace how upstream termination and pending promises are handled. Done would require a clearly documented or supported way to stop processing when QUERY_LIMIT_MESSAGE appears.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.