ruby / ruby/reline

Arrow key inputs are ignored on ENABLE_VIRTUAL_TERMINAL_INPUT enabled windows console

Open
#914 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
311
Forks
99
Avg merge
6d 4h
Merged PRs (30d)
1

Description

Symptoms similar to those described in #903 also occur in mswin/mingw.
ENABLE_VIRTUAL_TERMINAL_INPUT is normally disabled, but it may be enabled by other processes sharing the console, and it is possible that it is not reset due to a crash or other issue.

Step to reproduce

in irb

Reline::IOGate.instance_exec do
  m = "\0"*4; @GetConsoleMode.call(@hConsoleInputHandle, m);
  @SetConsoleMode.call(@hConsoleInputHandle, m.unpack1("L") | 0x200)
end

Or, for more flexibility,

irb(main):001> Reline::IOGate.class
=> Reline::Windows
irb(main):002* def set_vt_input(f)
irb(main):003*   Reline::IOGate.instance_exec do
irb(main):004*     m = "\0"*4; @GetConsoleMode.call(@hConsoleInputHandle, m);
irb(main):005*     mode = m.unpack1("L")
irb(main):006*     puts "old mode: #{mode.to_s(16)}"
irb(main):007*     new_mode = (mode & ~0x200) | (f ? 0x200 : 0)
irb(main):008*     @SetConsoleMode.call(@hConsoleInputHandle, new_mode)
irb(main):009*     puts "new mode: #{new_mode.to_s(16)}"
irb(main):010*   end
irb(main):011> end
=> :set_vt_input
irb(main):012> set_vt_input(true)
old mode: 1f7
new mode: 3f7
=> nil
irb(main):013> # => cursor movement using the arrow keys is disabled
irb(main):014> set_vt_input(false)
old mode: 3f7
new mode: 1f7
=> nil
irb(main):015> # => cursor movement using the arrow keys is enabled
Under the hood

Reline::Windows converts input key events to the output of getwch() rather than VT sequences (presumably to preserve the specifications from the early stages of development).
When ENABLE_VIRTUAL_TERMINAL_INPUT is enabled, Reline receives input that can be represented as VT sequences as characters rather than events, so no conversion to getwch() code takes place.
Since Reline::Windows bases its key bindings on getwch() codes, it cannot recognize input in the form of VT sequences.

Options for a Solution
  1. Add key bindings for VT sequences to Reline::Windows
  2. Stop using the getwch() code and have Reline::Windows convert key events to VT sequences

Although option 2 introduces more changes, since WSL coexists in Windows environments, this approach is preferable because it allows for the use of a common configuration.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Reline::Windows input handling and the Reline::IOGate console-mode calls described in the reproduction. Reproduce the issue in irb with ENABLE_VIRTUAL_TERMINAL_INPUT enabled, then verify that arrow-key cursor movement works in that mode while preserving the existing Windows and WSL behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.