IRB initialization changes global `$stdout` encoding, causing `Encoding::UndefinedConversionError`

Related Issue

This issue is related to #1048 (IRB leaking out Reline changes globally). In addition to Reline settings, IRB also leaks $stdout, $stdin, and $stderr encoding changes globally.

Description

When IRB is initialized, it calls $stdout.set_encoding('UTF-8') (and similarly for $stdin and $stderr). This affects all code that runs after IRB initialization, not just IRB itself.

This causes Encoding::UndefinedConversionError when code tries to output ASCII-8BIT strings to $stdout.

Minimal Reproducible Example

Example 1: Using IRB.setup and IRB::Irb.new

require 'irb'

puts "Before: $stdout.external_encoding = #{$stdout.external_encoding.inspect}"

# IRB initialization
IRB.setup(nil)
IRB::Irb.new

puts "After: $stdout.external_encoding = #{$stdout.external_encoding.inspect}"

# This raises Encoding::UndefinedConversionError
ascii_str = "日本語".force_encoding('ASCII-8BIT')
puts ascii_str

Output

Before: $stdout.external_encoding = nil
After: $stdout.external_encoding = #<Encoding:UTF-8>
-e:12:in 'IO#write': "\xE3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

Example 2: Using binding.irb

This is a more common scenario that affects everyday debugging:

puts "Before: $stdout.external_encoding = #{$stdout.external_encoding.inspect}"

binding.irb  # Enter IRB session, then type 'exit' to continue

puts "After: $stdout.external_encoding = #{$stdout.external_encoding.inspect}"

ascii_str = "日本語".force_encoding('ASCII-8BIT')
puts ascii_str

Output

Before: $stdout.external_encoding = nil
irb(main):001> exit
After: $stdout.external_encoding = #<Encoding:UTF-8>
-e:8:in 'IO#write': "\xE6" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

This means any code that runs after binding.irb is affected. Users debugging their applications with binding.irb may encounter unexpected encoding errors in code that worked fine before the IRB session.

Real-World Impact

This issue affects users of the debug gem with irb_console true setting.

Scenario

1. User has ~/.rdbgrc with config set irb_console true
2. User runs rails notes (or any Rails command)
3. debug gem loads and initializes IRB due to the setting
4. IRB changes $stdout.external_encoding to UTF-8
5. rails notes reads files with Encoding::BINARY and tries to output
6. Encoding::UndefinedConversionError is raised for non-ASCII content

Stack Trace

IRB::Irb#initialize (irb.rb:91)
  ↓
IRB::Context#initialize (context.rb:95)
  ↓
IRB::RelineInputMethod#initialize (input-method.rb:261)
  ↓
IRB.set_encoding (init.rb:528)
  ↓
$stdout.set_encoding (init.rb:529)  ← Global change!

Root Cause

In lib/irb/init.rb, the set_encoding method changes global I/O streams:

def set_encoding(extern, intern = nil, override: true)
  Encoding.default_external = extern
  Encoding.default_internal = intern
  [$stdin, $stdout, $stderr].each do |io|
    io.set_encoding(extern, intern)  # ← This affects all code, not just IRB
  end
  # ...
end

Proposed Solution

Instead of modifying global $stdin, $stdout, $stderr, IRB could use duplicated I/O streams for its own operations:

def set_encoding(extern, intern = nil, override: true)
  Encoding.default_external = extern
  Encoding.default_internal = intern

  # Use duplicated streams for IRB, don't modify global ones
  @irb_stdin = $stdin.dup.tap { |io| io.set_encoding(extern, intern) }
  @irb_stdout = $stdout.dup.tap { |io| io.set_encoding(extern, intern) }
  @irb_stderr = $stderr.dup.tap { |io| io.set_encoding(extern, intern) }
  # ...
end

This way, IRB maintains its encoding consistency without affecting other code.

Environment

• Ruby: 3.3.6 / 3.4.7
• IRB: 1.16.0
• OS: macOS (LANG=ja_JP.UTF-8)

Start in lib/irb/init.rb at set_encoding, then trace its callers through IRB::Context and IRB::RelineInputMethod. Ensure IRB keeps its required encoding behavior without changing the global standard streams, and verify both IRB.setup/IRB::Irb.new and binding.irb no longer make the ASCII-8BIT output example fail.