python-trio / python-trio/trio

Idea: deterministic thread and I/O handling for certain tests

Open
#3,089 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

For testing, e.g. with Hypothesis, it can often be useful to run deterministically even at the cost of performance.

Using a MockClock(rate=0, autojump_threshold=0) and setting _ALLOW_DETERMINISTIC_SCHEDULING=True does the trick for tests that only use in-process Trio interfaces, but if you need to test with real network IO or multiple threads, then the variation in timing can change which tasks can be scheduled at each point, and we're nondeterministic again.

I recently saw a neat proof-of-concept that gets around this: whenever you would do real IO (i.e. not controlled by the MockClock), do the real IO but don't let reality control the timing. Instead, use the PRNG to choose how long that operation will take - if it finishes earlier, hold it until the chosen time; if it's not ready by then hold the rest of the program (including the clock) until the operation finishes.

Brief notes:

  • I think we'd want to patch trio.to_thread.run_sync, trio.from_thread.run, trio.from_thread.run_sync, trio.serve_listeners, trio.open_tcp_stream, trio.lowlevel.wait_readable, trio.lowlevel.wait_writable, and trio.open_signal_receiver to support this where possible, or explicitly error out otherwise.
  • It might also be interesting to automatically use an in-process network when a client and server are in the same process, instead of going over real OS sockets. This is very convenient but also I dislike silently testing something other than what the user expects.
  • I think this is too big a change to automatically activate whenever there's a suitable clock and deterministic scheduling is enabled, so we'll want some way to explicitly turn it on, too.

I'm not fully convinced that we should ship something like this, but it seems interesting enough to write up and investigate futher.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the deterministic scheduling setup described in the issue: MockClock, _ALLOW_DETERMINISTIC_SCHEDULING in src/trio/_core/_run.py, and the listed thread, socket, signal, and low-level I/O APIs. Investigate how real I/O timing could be controlled across these entry points, including the in-process network idea. Done would require a decided scope and explicit activation behavior, but the issue does not define an implementation plan or acceptance tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.