typelevel / typelevel/cats-effect
Figure out how to use polling systems outside of WSTP and `IO`
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.2k
- Forks
- 576
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 18
Description
Currently the only place you can materialize a PollingSystem is the work-stealing threadpool. Furthermore, all of our polling interfaces are currently in terms of IO and to get a hold of one you need an IORuntime.
Why do we care? Even if we are in IO on the WSTP, it might have the wrong kind of polling system installed, so we need some kind of fallback. Additionally, FS2 has promised Network and friends for any Async effect so we need to consider how to keep supporting that without having to maintain a handful of separate implementations that effectively do the same thing ...
I think these problems are somewhat orthogonal, but I suspect they are best solved jointly. Specifically we need:
-
a way to materialize a polling system on a "selector thread"—some sort of single-threaded event loop that just does I/O polling.
-
a way to use our polling interfaces from a non-
IOeffect
Controversial idea: for (1) just make a WSTP with one thread. For (2) keep using IO and use a Dispatcher to implement LiftIO[F] for arbitrary Async[F], similar to what Doobie does. This would work, and is arguably no different than using an NIO2 / Netty selector thread with CompletableFuture / ChannelFuture. Same deal: someone else's event loop with someone else's coroutine.
Follow-up to:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the PollingSystem, WSTP, IO, IORuntime, Dispatcher, LiftIO, and Async interfaces, then review the linked follow-up pull request 3332. The work is complete when polling can be materialized on a selector thread and the polling interfaces can be used from a non-IO effect without maintaining separate implementations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100