`test_control_mode_stdout_preserves_non_ascii_output` polls `select` on a buffered stream

Start in tests/test_control_mode.py at test_control_mode_stdout_preserves_non_ascii_output and inspect the select/readline loop. Replace the select-based wait with the proposed queue.Queue reader thread and deadline, then run the focused test, including with the UTF-8 encoding regression removed to confirm UnicodeDecodeError coverage remains.

Filed against tmux-python/libtmux v0.62.0, tmux 3.7b. Flaked twice in CI on unrelated pull requests, on different tmux matrix cells, both times passing on re-run.

What happens

test_control_mode_stdout_preserves_non_ascii_output bounds its wait with select.select() on a buffered stream and then consumes with readline(), so readiness and readability are measured against two different buffers.

tests/test_control_mode.py#L88-L94

ControlMode builds its subprocess with text=True, encoding="utf-8", so stdout is a TextIOWrapper over a BufferedReader:

src/libtmux/_internal/control_mode.py#L83-L90 and #L99

select calls fileno() and sees only the kernel pipe. One readline() drains up to 8192 bytes off that pipe into userspace and returns the first line; the rest of the reply sits in the decoder where select cannot see it, and the descriptor is now empty.

tmux frames each command's control-mode output with %begin / %end guards written in one burst, so the whole reply arrives together:

cmd-queue.c#L825-L832

Observed timeline from a raw os.read on the same descriptor:

t=0.0002  read(68)   %begin ... \n%end ... \n%session-changed $0 s\n
t=0.0003  read(50)   %begin ... \n\xe2\x90\x9e\n%end ... \n
t=0.4619  read(137)  %output %0 ...

The separator is fully buffered by the first or second readline(). What normally rescues the test is that third read: unrelated %output from the pane's shell painting its prompt re-arms select, and the loop then serves its remaining lines out of the userspace buffer. The test passes for a reason unrelated to the data it asserts on.

Reproduction

Deterministic once the read loop starts after the prompt-paint burst has landed, which is what a loaded CI runner produces. Injecting a stall between ControlMode.__enter__ and the loop, with everything else verbatim:

delay=0.0   0/4 failed
delay=0.15  1/4 failed
delay=0.3   3/4 failed

Proof the data was in hand at the moment select reported not-ready:

select NOT ready at iter=1 (last read = '%begin 1785458429 355 1')
  readline #0 returned in 7us  -> '␞'
  readline #1 returned in 3us  -> '%end 1785458429 355 1'
  readline #2 BLOCKED (fd empty)

select burned its full one-second budget while the answer was seven microseconds away.

CI signature

tests/test_control_mode.py:91: in test_control_mode_stdout_preserves_non_ascii_output
    assert ready, "timed out waiting for control-mode output"
E   AssertionError: timed out waiting for control-mode output
E   assert []
        line       = '%begin 1785370231 290 0\n'
        ready      = []

line is the loop variable holding the last line successfully read, so in both failures the test read %begin and the next select timed out. That is the signature of "read one line, then the descriptor went quiet", not of a slow tmux.

Ruled out, with evidence

• The locale.setlocale(LC_CTYPE, "C") the test performs. Removing it changes nothing: 4/4 failures at delay=0.3.
• tmux version. The mechanism is entirely Python-side. Two flakes landing on two of eight matrix cells is what randomness looks like.
• bufsize on Popen. Buffering mode is a write-side setting; measured no effect for 0, 1, or default.

What a fix needs

The library code is correct and should not change. text=True, encoding="utf-8" is deliberate — it is the regression this test exists to guard.

1. Drop select; read on a thread with a deadline. A reader thread pushes lines onto a queue.Queue, the test polls the queue with a timeout. Nothing straddles two buffers. Verified that a threaded reader still catches the original regression: with encoding="utf-8" removed it raises UnicodeDecodeError, so the coverage is preserved.
2. Or select on the raw descriptor via ctl.stdout.buffer.raw and decode in the test. Correct, but the test then owns partial-line handling and decoding.

Option 1 is preferred: the only reason select is present is to bound the wait, and a thread with a deadline does that without introducing a second buffer.

Related

tmux wait-for is the general answer for authored commands and is already wrapped as Server.wait_for() (src/libtmux/server.py#L617, cmd-wait-for.c#L34-L38), but it does not apply here: this test waits on control-mode protocol output rather than on a shell command it authored.