prompt-toolkit / prompt-toolkit/python-prompt-toolkit
prompt_toolkit deadlocks eventloop during flush_stdout
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 815
- PR merge metrics
- No merged PRs in 30d
Description
I've been using prompt_toolkit within a PTY for my package madbg, and it has been working fine, until I recently got a new computer screen. Since I've had it, my prompt_toolkit app would hang from time to time. You might be wondering how a new screen might cause a sporadic bug in prompt_toolkit... And the answer is that the new screen is somewhat larger than my previous one.
I'm running a fullscreen prompt_toolkit application, so during _redraw, flush_stdout attempts to blockingly write a chunk of data that correlates to the size of my terminal, which in turn correlates to the size of my screen. The hang occurs becuase for some terminal sizes, prompt_toolkit will attempt to write more bytes than the PTY buffer has available. According to this comment by @jonathanslenders , this is the intended behavior - in most cases, this call would shortly block until some other thread or process would read from the other end of the pipe. But in my case, it's the same eventloop that handles the reads and writes of both the slave fd and the master fd of the PTY, so the blocking call in flush_stdout blocks forever 😢
I suspect that this might happen in many of the cases where prompt_toolkit is used within a PTY (and not with a regular pipe), because AFAICT, the PTY buffer size in the linux kernel is far smaller than that of a pipe (4096 bytes?), and it is more likely for one thread to be responsible for both ends of a PTY than of a regular pipe.
As a workaround until this is resolved, I can use a separate thread for my prompt_toolkit app, but I'd like to help solving this in prompt_toolkit. I'm not sure what would be a good solution though, as flush_stdout is called by PlainTextOutput.flush and Vt100_Output.flush, which are both non-async and are called from non-async functions...
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/prompt_toolkit/output/flush_stdout.py and trace how PlainTextOutput.flush and Vt100_Output.flush call it. Reproduce the hang with a fullscreen prompt_toolkit application running in a PTY whose output exceeds the PTY buffer. Done means the event loop no longer deadlocks during this flush path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100