zip + uncons + concurrency = scope lookup failure
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.5k
- Forks
- 636
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
Quoting @armanbilge:
The problem seems to be running the tail of a
zipped Stream.concurrently.
I ran into this issue before here: https://github.com/typelevel/fs2/issues/3081#issuecomment-1398495481
IIUC Arman worked around that problem for hold1 by moving the .pull.uncons into the concurrent process. However the core issue still remains: if you do stream.pull.uncons and then concurrently process the tail, things will blow up if stream happens to contain any kind of zipping. Possibly other operators that are implemented with stepLeg have the same issue?
I ran into this again while trying to use Pull.extendScopeTo for extending resource lifetimes across async boundaries. That actually seems to work, but requires stream.pull.peek or any other unconsing variation, which means it's incompatible with zipped streams:
input1.pull.peek.flatMap{
case Some((_, stream)) =>
Pull.extendScopeTo(asyncBufferThing(stream)).flatMap(s => Pull.output1(s))
case None =>
Pull.done
}
.stream
.flatten
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure from the linked Scastie example, then inspect Pull.uncons or Pull.peek on zipped streams followed by concurrently. Read the related discussion in issues 3081 and the operators implemented with stepLeg; done means this scope lookup failure is reproduced by a regression test and no longer occurs for the reported composition.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100