python-trio / python-trio/trio

Can we make abort functions more composable?

Open
#896 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design discussion potential API breaker
Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

In chat today, @njsmith gave a neat example of how to build a resource-aware job scheduler using trio.Event. He mentioned that doing the bookkeeping in the task that's doing the wakeup, rather than in the task that's being woken up, seems broadly essential to avoiding race conditions (which I agree with).

I think there's a rough edge with respect to how cancellation handling fits into this, though:

  • If you block with wait_task_rescheduled, your abort_fn runs synchronously with the cancellation being delivered. So if the waking task sees that you're still registered, you won't be cancelled before you're woken up; and if you do get cancelled, the waking task won't see you as still eligible to be woken up. This is great. It means the most natural way to handle cancellation is also relatively race-proof.
  • But if you block by calling some other abstraction, such as trio.Event.wait(), the only way you can do cleanup on cancellation is by catching the Cancelled exception after it's injected. Moreover, it's possible that a task gets cancelled, but that before it next runs to deliver the cancellation, it also is woken up due to the resource it was waiting for becoming available. Now we're back to writing bookkeeping logic inside the woken task. This is less great.

It seems to me that we might be well served by having a way of effectively composing abort functions. That is, when I say await event.wait(), I should have the option of designating some code that will run synchronously with a cancellation of event.wait().

Doing this in a way that lets the outer logic block the cancellation (i.e., exposing the full abort_fn power) is very tricky. Doing it in a way that only supports supplying additional code to run when the cancellation succeeds is probably easier though. Do folks have thoughts on whether this is desirable, independent of how it might best be implemented?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the cancellation behavior around wait_task_rescheduled, abort_fn, trio.Event.wait(), and Cancelled described in the issue and its linked discussion. The issue does not name files or tests, and a contribution would first need an agreed design and clear completion criteria for composing cancellation handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.