typelevel / typelevel/fs2

zip + uncons + concurrency = scope lookup failure

Open
#3,478 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Scala
Stars
2.5k
Forks
636
Avg merge
2d 4h
Merged PRs (30d)
7

Description

Quoting @armanbilge:

The problem seems to be running the tail of a zipped Stream .concurrently.

I ran into this issue before here: https://github.com/typelevel/fs2/issues/3081#issuecomment-1398495481

IIUC Arman worked around that problem for hold1 by moving the .pull.uncons into the concurrent process. However the core issue still remains: if you do stream.pull.uncons and then concurrently process the tail, things will blow up if stream happens to contain any kind of zipping. Possibly other operators that are implemented with stepLeg have the same issue?

I ran into this again while trying to use Pull.extendScopeTo for extending resource lifetimes across async boundaries. That actually seems to work, but requires stream.pull.peek or any other unconsing variation, which means it's incompatible with zipped streams:

input1.pull.peek.flatMap{
  case Some((_, stream)) =>
    Pull.extendScopeTo(asyncBufferThing(stream)).flatMap(s => Pull.output1(s))
  case None =>
    Pull.done
}
.stream
.flatten

https://scastie.scala-lang.org/56CQGK8uTzecITHSwmO0AQ

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure from the linked Scastie example, then inspect Pull.uncons or Pull.peek on zipped streams followed by concurrently. Read the related discussion in issues 3081 and the operators implemented with stepLeg; done means this scope lookup failure is reproduced by a regression test and no longer occurs for the reported composition.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.