prompt-toolkit / prompt-toolkit/python-prompt-toolkit

filedescriptor still out of range in select

Open
#1,702 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10.6k
Forks
815
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm still having some problems with select.select raising an exception "ValueError: filedescriptor out of range in select()" in src/prompt_toolkit/input/posix_utils.py", line 72 if the file descriptor number is too large (>1023). This happens when passing in a pipe instead of stdin during testing. It seems to be similar to problems that were already discussed in #354.

What makes this especially problematic is that the exception occurs inside the prompt loop which results in the prompt (and pytest) hanging indefinitely. A simple test to reproduce the problem:

import os
from typing import NamedTuple

import pytest

from prompt_toolkit import PromptSession
from prompt_toolkit.input import create_pipe_input
from prompt_toolkit.input.base import PipeInput
from prompt_toolkit.output import DummyOutput


@pytest.fixture
def prompt():
    file_descriptors = []
    try:  # if an "OSError: Too many open files" is thrown, you may need to adjust the fp limit (`ulimit -n`)
        for _ in range(512):  # this should be enough to create a file descriptor > 1023
            read_fd, write_fd = os.pipe()
            file_descriptors.extend([read_fd, write_fd])
        with create_pipe_input() as input_:
            session = PromptSession(input=input_, output=DummyOutput())
            yield Prompt(session, input_)
    finally:
        for fd in file_descriptors:
            os.close(fd)


class Prompt(NamedTuple):
    session: PromptSession
    input: PipeInput


def test_too_many_open_files(prompt):
    prompt.input.send_text(f"foobar\n")
    # the prompt will hang if `select.select` is used because it causes an exception "filedescriptor out of range"
    prompt.session.prompt("foobar")

As discussed in #354, it is possible to easily fix this by replacing select.select with select.poll. I have added a patch which seemingly fixes this:
select_poll_patch.txt

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/prompt_toolkit/input/posix_utils.py at line 72 and reproduce the failure with the provided test using create_pipe_input and a high-numbered file descriptor. Review the linked select_poll_patch.txt and verify the prompt no longer hangs or raises the filedescriptor-out-of-range exception when the test completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.