Python 3.12 test failures (Too many open files)

Breaking this issue out from https://github.com/prompt-toolkit/python-prompt-toolkit/pull/1714#issuecomment-1411115166.

The follow tests fail when run on Python 3.12.0a4.

FAILED tests/test_cli.py::test_emacs_arguments_for_all_commands - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_emacs_kill_ring - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_emacs_selection - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_emacs_insert_comment - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_emacs_record_macro - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_emacs_nested_macro - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_prefix_meta - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_bracketed_paste - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_cursor_movements - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_operators - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_text_objects - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_digraphs - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_block_editing - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_block_editing_empty_lines - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_visual_line_copy - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_visual_empty_line - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_character_delete_after_cursor - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_character_delete_before_cursor - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_character_paste - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_temp_navigation_mode - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_vi_macros - OSError: [Errno 24] Too many open files
FAILED tests/test_cli.py::test_accept_default - OSError: [Errno 24] Too many open files
FAILED tests/test_completion.py::test_pathcompleter_completes_directories_with_only_directories - OSError: [Errno 24] Too many open files: '/tmp/tmpweazk1ay/subdir'
FAILED tests/test_completion.py::test_pathcompleter_get_paths_constrains_path - OSError: [Errno 24] Too many open files: '/tmp/tmphxggdzn6/subdir'
FAILED tests/test_history.py::test_in_memory_history - OSError: [Errno 24] Too many open files
FAILED tests/test_history.py::test_file_history - OSError: [Errno 24] Too many open files
FAILED tests/test_history.py::test_threaded_in_memory_history - OSError: [Errno 24] Too many open files
FAILED tests/test_key_binding.py::test_feed_simple - OSError: [Errno 24] Too many open files
FAILED tests/test_key_binding.py::test_feed_several - OSError: [Errno 24] Too many open files
FAILED tests/test_key_binding.py::test_control_square_closed_any - OSError: [Errno 24] Too many open files
FAILED tests/test_key_binding.py::test_common_prefix - OSError: [Errno 24] Too many open files
FAILED tests/test_key_binding.py::test_previous_key_sequence - OSError: [Errno 24] Too many open files
FAILED tests/test_shortcuts.py::test_print_container - OSError: [Errno 24] Too many open files
ERROR tests/test_history.py::test_threaded_file_history - OSError: [Errno 24] Too many open files: '/tmp/pytest-of-carl/pytest-3'

The full output for test_emacs_arguments_for_all_commands is in that other issue. I'm happy to post the output from any of these individual tests, but I figured I wouldn't overload the issue with off of the output unless needed.

Start by running the listed tests in tests/test_cli.py, tests/test_completion.py, tests/test_history.py, tests/test_key_binding.py, and tests/test_shortcuts.py under Python 3.12. Read the full output for test_emacs_arguments_for_all_commands in the linked issue and identify the source of the open-file exhaustion. Done means the affected tests pass without OSError: [Errno 24].