ruby / ruby/irb

UndefinedConversionError when LANG and default encoding are different

Open
#1,023 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
478
Forks
158
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Description

In this situation, IRB raises UndefinedConversionError

[tomoya.ishida:3.2]% echo "IRB.conf[:USE_MULTILINE]=false" > irbrc
[tomoya.ishida:3.2]% IRBRC=irbrc LANG=en_US.ASCII ruby -Eutf-8:utf-8 -e "binding.irb"              
irb(main):001> Readline::VERSION
=> "8.2"
irb(main):002> \343\201\202\343\201\202\343\201\202
/Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `write': "\xE3" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to US-ASCII (Encoding::UndefinedConversionError)
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `puts'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `block in save_history'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:73:in `open'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:73:in `save_history'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1033:in `ensure in run'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1034:in `run'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1587:in `irb'
	from <internal:prelude>:5:in `irb'
	from -e:1:in `<main>'
/Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/context.rb:628:in `strip': invalid byte sequence in US-ASCII (Encoding::CompatibilityError)
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/context.rb:628:in `parse_command'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1134:in `build_statement'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1122:in `block in each_top_level_statement'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1119:in `loop'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1119:in `each_top_level_statement'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1042:in `eval_input'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1023:in `block in run'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1022:in `catch'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1022:in `run'
	from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1587:in `irb'
	from <internal:prelude>:5:in `irb'
	from -e:1:in `<main>'

Similar thing might happen in a situation described in https://github.com/ruby/reline/issues/755#issuecomment-2396392234

Result of irb_info

Ruby version: 3.2.2
IRB version: irb 1.14.1 (2024-09-25)
InputMethod: ReadlineInputMethod with ext/readline 8.2
Completion: RegexpCompletor
.irbrc paths: irbrc, /Users/tomoya.ishida/github/ruby/3.2/irbrc
RUBY_PLATFORM: arm64-darwin22
LANG env: en_US.ASCII
East Asian Ambiguous Width: 1

Terminal Emulator

Terminal.app

Setting Files

irbrc

IRB.conf[:USE_MULTILINE]=false

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported command with LANG=en_US.ASCII and -Eutf-8:utf-8, then inspect irb/history.rb around save_history and irb/context.rb around parse_command. Trace how the differing encodings reach history writing and command parsing. Done means the reported interaction no longer raises either encoding error.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.