UndefinedConversionError when LANG and default encoding are different
Open
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 478
- Forks
- 158
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Description
In this situation, IRB raises UndefinedConversionError
[tomoya.ishida:3.2]% echo "IRB.conf[:USE_MULTILINE]=false" > irbrc
[tomoya.ishida:3.2]% IRBRC=irbrc LANG=en_US.ASCII ruby -Eutf-8:utf-8 -e "binding.irb"
irb(main):001> Readline::VERSION
=> "8.2"
irb(main):002> \343\201\202\343\201\202\343\201\202
/Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `write': "\xE3" to UTF-8 in conversion from ASCII-8BIT to UTF-8 to US-ASCII (Encoding::UndefinedConversionError)
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `puts'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:82:in `block in save_history'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:73:in `open'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/history.rb:73:in `save_history'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1033:in `ensure in run'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1034:in `run'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1587:in `irb'
from <internal:prelude>:5:in `irb'
from -e:1:in `<main>'
/Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/context.rb:628:in `strip': invalid byte sequence in US-ASCII (Encoding::CompatibilityError)
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb/context.rb:628:in `parse_command'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1134:in `build_statement'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1122:in `block in each_top_level_statement'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1119:in `loop'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1119:in `each_top_level_statement'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1042:in `eval_input'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1023:in `block in run'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1022:in `catch'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1022:in `run'
from /Users/tomoya.ishida/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.14.1/lib/irb.rb:1587:in `irb'
from <internal:prelude>:5:in `irb'
from -e:1:in `<main>'
Similar thing might happen in a situation described in https://github.com/ruby/reline/issues/755#issuecomment-2396392234
Result of irb_info
Ruby version: 3.2.2
IRB version: irb 1.14.1 (2024-09-25)
InputMethod: ReadlineInputMethod with ext/readline 8.2
Completion: RegexpCompletor
.irbrc paths: irbrc, /Users/tomoya.ishida/github/ruby/3.2/irbrc
RUBY_PLATFORM: arm64-darwin22
LANG env: en_US.ASCII
East Asian Ambiguous Width: 1
Terminal Emulator
Terminal.app
Setting Files
irbrc
IRB.conf[:USE_MULTILINE]=false
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported command with LANG=en_US.ASCII and -Eutf-8:utf-8, then inspect irb/history.rb around save_history and irb/context.rb around parse_command. Trace how the differing encodings reach history writing and command parsing. Done means the reported interaction no longer raises either encoding error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100