typelevel / typelevel/cats-effect

Figure out if and how to use io_uring inter-ring messages for polling interruption

Open
#3,684 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
2.2k
Forks
576
Avg merge
2d 11h
Merged PRs (30d)
18

Description

An important feature of the polling system is the ability to interrupt a poller while it is polling. This is necessary to wake up a worker thread e.g. when there is work available in the external queue.

https://github.com/typelevel/cats-effect/blob/58f695fbc25a2f7bef5d04427ec5d03e94212e8d/core/jvm-native/src/main/scala/cats/effect/unsafe/PollingSystem.scala#L56

io_uring now has the ability to directly send messages between rings. This seems like a great way to implement this interruption, since sending a message to a ring would cause it to wake up immediately.

https://man7.org/linux/man-pages/man3/io_uring_prep_msg_ring.3.html

However, sending said message requires having access to a ring to send the message from. This opens a few questions:

  1. should the PollingSystem#interrupt API be given access to the poller of its current thread, if available?

  2. if we are not on a WSTP thread and we don't have a thread-local ring, what should we do? This is a common case, since work for the external queue will originate from external threads.

  3. even if we do have a thread-local ring we can use to send the message, should we bypass the usual submission mechanism? otherwise it may take up to 64 iterations of the worker loop until the interrupt even gets submitted, which is a lot of latency. seems kinda dumb

So, tough questions. We could implement this today, without access to the thread-local ring, by keeping a separate collection of rings used only for sending interrupt messages. This is is necessary anyway to support sending interrupts from external threads.

Follow-up to:

cc @antoniojimeneznieto

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with PollingSystem.scala at the linked interrupt API and read the io_uring_prep_msg_ring documentation. Review the follow-up context from pull request #3332 and the three unanswered design questions about thread-local rings, external threads, and submission latency. Done means documenting and implementing an agreed approach for interrupt delivery.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.