typelevel / typelevel/cats-effect
Unsafe run sync in CE3
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.2k
- Forks
- 576
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 18
Description
I'm trying to migrate my library (FP-friendly wrapper for hbase-client) from CE2 to CE3. Some part of the library is built on top of an impure callback-based Java API. This API has a restriction: actions inside the callbacks should be executed synchronously on a caller thread. Here you can see an interface I'm trying to implement: AdvancedScanResultConsumer. Note the Javadoc of ScanController: "... you should only call the suspend() or terminate() inside onNext or onHeartbeat method. A IllegalStateException will be thrown if you call them at other places."
In CE2, I used f.toIO.unsafeRunSync() for this purpose. As far as I can see, it runs all the actions on the current thread, and everything works fine.
In CE3, I tried to solve it using Dispatcher.unsafeRunSync and IO.evalOn. But as far as I get it, dispatcher.unsafeRunSync(f.evalOn(ec)) runs evaluation on the default compute pool while blocking a caller thread and then shifts execution to ec. But the caller thread is the thread on which I'd like to execute actions described by f.
There's a hacky solution using both Dispatcher.unsafeRunSync and IO.syncStep. If I reorder actions inside the callback placing synchronous actions (which must be executed on the caller thread) before the asynchronous one, then the following code works:
import cats.effect.IO
import cats.effect.std.{Dispatcher, Queue}
import org.apache.hadoop.hbase.client.{AdvancedScanResultConsumer, Result}
class ScanResultConsumerImpl(dispatcher: Dispatcher[IO], queue: Queue[IO, Array[Result]]) extends AdvancedScanResultConsumer {
override def onNext(results: Array[Result], controller: ScanController): Unit = {
unsafeRun(
for {
// controller.suspend() must be called from the exact same thread as the onNext itself
_ <- IO.delay(controller.suspend())
// Other synchronous steps...
_ <- queue.offer(value)
} yield ()
)
private def unsafeRun[A](f: IO[A]): Unit =
f.syncStep.unsafeRunSync() match {
case Right(result) => result
case Left(asyncF) => dispatcher.unsafeRunSync(asyncF)
}
// The remainder is skipped for brevity
}
It would be great to have a built-in mechanism to handle such use cases in CE3.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the issue's examples of Dispatcher.unsafeRunSync, IO.evalOn, and IO.syncStep, together with the AdvancedScanResultConsumer callback contract. Determine the intended built-in mechanism for preserving caller-thread execution of synchronous callback actions, and define completion as a supported CE3 API that handles the example without the ordering hack.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100