New REPL (pyrepl) calls PyOS_InputHook with the terminal in raw mode, so output from input-hook callbacks loses carriage returns

Bug report

Bug description:

In the new REPL (_pyrepl), the terminal is put into raw mode with output post-processing (OPOST) disabled while the reader waits for input, and PyOS_InputHook is invoked from within that raw-mode read loop. As a result, any text an input-hook callback writes to the terminal is emitted with bare \n and no \r, so multi-line output "staircases" down and to the right instead of returning to column 0.

This is a regression from the classic (readline-based) REPL, which kept OPOST on and so let input-hook callbacks print normally. Setting PYTHON_BASIC_REPL=1 restores the old, correct behavior.

Input hooks are how GUI toolkits keep their event loops alive at the interactive prompt (Tkinter, and via their bindings PyQt/PySide, etc.). Any callback that runs during event processing and writes to the terminal — a logged warning, a traceback, a print — is affected. Confirmed with matplotlib (QtAgg) and PyVista/pyvistaqt, both purely through the standard PyOS_InputHook path.

Minimal reproducer (pure stdlib, no third-party packages)

# repro.py  --  run with:  python -i repro.py
# Then just sit at the >>> prompt for a couple of seconds and watch the output.
import ctypes, sys, time

_last = [0.0]
_HOOK = ctypes.CFUNCTYPE(ctypes.c_int)

def _hook():
    now = time.time()
    if now - _last[0] > 1.0:            # print ~once/sec while waiting at the prompt
        _last[0] = now
        sys.stderr.write("line1\nline2\nline3\n")
        sys.stderr.flush()
    return 0

_cb = _HOOK(_hook)
ctypes.c_void_p.in_dll(ctypes.pythonapi, "PyOS_InputHook").value = \
    ctypes.cast(_cb, ctypes.c_void_p).value
print("Sit at the >>> prompt and watch how line1/line2/line3 are laid out.")

Run it and leave the prompt idle:

python -i repro.py

Actual (new REPL) — the lines march diagonally across the screen; each new line begins at the column where the previous one ended (bare \n, no \r):

>>> line1
         line2
              line3

Expected (and what PYTHON_BASIC_REPL=1 python -i repro.py produces):

>>> line1
line2
line3

Captured raw bytes make the difference unambiguous (pty, TERM=xterm-256color):

new REPL:                line1<LF>line2<LF>line3<LF>          # OPOST off, no CR
PYTHON_BASIC_REPL=1:     line1<CR><LF>line2<CR><LF>line3<CR><LF>

Root cause

_pyrepl.unix_console.UnixConsole.prepare() clears OPOST when entering raw mode:

# Lib/_pyrepl/unix_console.py  (prepare)
self.__svtermstate = tcgetattr(self.input_fd)
raw = self.__svtermstate.copy()
...
raw.oflag &= ~(termios.OPOST)

and Lib/_pyrepl/reader.py calls the input hook while still in that raw state:

# Lib/_pyrepl/reader.py
input_hook = self.console.input_hook
if input_hook:
    try:
        input_hook()
    except Exception:
        pass

So the callback runs with OPOST disabled and its output is not translated (\n is not mapped to \r\n).

Suggested fix

pyrepl needs OPOST cleared for its own cursor/screen rendering, so it can't simply leave it on. But the input-hook call is exactly when pyrepl is not drawing, and hook callbacks legitimately expect normal cooked-mode output (as they had under the readline REPL). UnixConsole.prepare() already saves the original terminal state in self.__svtermstate (and restore() re-applies it via self.__input_fd_set(self.__svtermstate)), so the fix can be localized: around the input_hook() call, temporarily restore the original output flags — or at least OPOST/ONLCR — then re-enter raw mode. A fix here benefits every input-hook provider (Tkinter, PyQt, PySide, …) rather than requiring each GUI binding to work around it individually.

I have verified this fix works in manual testing in https://github.com/pyvista/pyvistaqt/pull/843 as well.

Related (both closed)

• gh-119842 — "Honor PyOS_InputHook in the new REPL" (got the hook called at all)
• gh-121497 — "Pyrepl doesn't respect correctly the history with input hook set" (a separate display bug with hooks)

Neither addresses the terminal output-mode problem described here. The relevant code (prepare() clearing OPOST, input_hook returning posix._inputhook) is unchanged on main as of filing.

Disclosure

An AI tool (Claude) was used to help investigate and write up this report. I reviewed it in full and take responsibility for its content. The behavior, the minimal reproducer, and the PYTHON_BASIC_REPL=1 contrast were verified on a real CPython 3.13.9 build, and the unix_console.py/reader.py references were checked against the current main source.

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

• gh-153389

Comece por Lib/_pyrepl/unix_console.py, especialmente UnixConsole.prepare() e restore(), e depois acompanhe a chamada do input hook em Lib/_pyrepl/reader.py. Reproduza o problema com o script fornecido python -i repro.py e compare o novo REPL com PYTHON_BASIC_REPL=1; considera-se concluído quando a saída do hook mantém os retornos de carro normais sem interromper a renderização do pyrepl.