laziness and look-ahead
- Dominant language
- JavaScript
- Stars
- 3.4k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
I think I undestand the lazy aspect of highland : an element of a stream is read only when it is pulled.
with the current implementation it seems that in a graph like
```
_([1,2,3]).map(op1).map(op2).consume()
```
the execution path will be :
- first op2(op1(1))
- then op2(op1(2))
- then op2(op1(3))
even with async operations
```
_([1,2,3]).flatMap(async1).flatMap(async2).consume()
```
the execution will do
- async2 of async1 of 1
- then async2 of async1 of 2
- then async2 of async1 of 3
My question is : is that considered a feature of highland to do things one at a time with flatMap, or should we expect that one day, there will be look-ahead and that the path here could become for example
- async1 of 1
- then async1 of 2
- then async2 of async1 of 1
- then async2 of async1 of 2
- then async2 of async1 of 3
I know that things like parallel or merge can buffer things, or randomize the order of things but I am wondering what kind of guarantee we have for the order or processing if a mutable object flows inside the graph is is modified by operations.
Visually, if I look at a stream graph like a matrix
```
oooooooooooooo...
oooooooooooooo...
oooooooooooooo...
oooooooooooooo...
```
where the horizontal-axis are the stream elements (potentially infinite) and the vertical-axis the operations, the flow seem to be the following
```
xooooooooooooo...
xooooooooooooo...
xooooooooooooo...
xooooooooooooo...
```
then
```
xxoooooooooooo...
xxoooooooooooo...
xxoooooooooooo...
xxoooooooooooo...
```
etc. the picture is like a vertical going one element after the other.
could it one day in highland become
```
xxoooooooooooo...
xooooooooooooo...
oooooooooooooo...
oooooooooooooo...
```
then
```
xxxooooooooooo...
xxoooooooooooo...
xooooooooooooo...
oooooooooooooo...
```
then
```
xxxxoooooooooo...
xxxooooooooooo...
xxoooooooooooo...
xooooooooooooo...
```
etc. the picture is more like a flood coming from the upper left corner.
I hope that I am being clear ; I would like to know the design ideas of the project on this point if you can share your thoughts on the matter. Thanks !
Contributor guide
Research direction
Start by reviewing the implementation and documentation for flatMap, consume, parallel, and merge. Trace how async values move through chained streams and inspect any existing ordering tests. Done means the project’s look-ahead behavior and ordering guarantees for mutable values are clearly decided and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100