Bundler unexpectly Kernel.exec on require 'bundler/setup' and loses all command-line flags and CWD

Describe the problem as clearly as you can

This can be reproduced with:

$ chruby 3.3.5
$ bundle config --global path vendor/bundle
$ gem uni bundler:2.4.19

$ git clone git@github.com:eregon/yjit-bench.git
$ cd yjit-bench
$ git checkout bundler-bug
$ ruby --yjit -Iharness-warmup $PWD/benchmarks/activerecord/benchmark.rb

gives:

$ ruby --yjit -Iharness-warmup $PWD/benchmarks/activerecord/benchmark.rb

Starting activerecord benchmark with:
PID: 613108
$LOAD_PATH
/home/eregon/code/yjit-bench/harness-warmup
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0/x86_64-linux
Dir.pwd: /home/eregon/code/yjit-bench
RUBY_DESCRIPTION: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [x86_64-linux]

ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [x86_64-linux]
Command: bundle check 2> /dev/null || bundle install
The Gemfile's dependencies are satisfied

Starting activerecord benchmark with:
PID: 613108
$LOAD_PATH
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0/x86_64-linux
Dir.pwd: /home/eregon/code/yjit-bench/benchmarks/activerecord
RUBY_DESCRIPTION: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]

/home/eregon/code/yjit-bench/harness/harness.rb:3:in `<top (required)>': should not reach here (RuntimeError)
	from <internal:/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
	from <internal:/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
	from /home/eregon/code/yjit-bench/harness/loader.rb:4:in `<top (required)>'
	from /home/eregon/code/yjit-bench/benchmarks/activerecord/benchmark.rb:9:in `require_relative'
	from /home/eregon/code/yjit-bench/benchmarks/activerecord/benchmark.rb:9:in `<main>'

So there we can see:

• The PID is the same and Bundler is calling Kernel.exec unexpectedly, which causes:
• The first $LOAD_PATH entry has been lost (/home/eregon/code/yjit-bench/harness-warmup from -Iharness-warmup)
• Which means the wrong harness is loaded and we get to that should not reach here (RuntimeError)
• The current working dir has changed. This means relative paths are reinterpreted differently and incorrectly, hence $PWD/benchmarks/activerecord/benchmark.rb above and this workaround: https://github.com/eregon/yjit-bench/commit/fb09723ca9bab9f6f4d867d7291b182014c122b9#diff-286ad695324de1cdc1615695f4f4158b17dc9da8229989a28f12b9cacc1eec64
• The second one lost the --yjit option and so no JIT is enabled

You can imagine how this can be a "what the hell" moment when the user runs one command but Bundler silently exec's, which means running the command a second time and loses a bunch of critical state on require "bundler/setup".

Luckily this can be worked around by gem install bundler:2.4.19.
So part of the issue is that with the bundle config as bundle config --global path vendor/bundle the locked bundler version gets installed to benchmarks/activerecord/vendor/bundle/ruby/3.3.0/gems/bundler-2.4.19.

But the bigger issue is I would never expect any require to call Kernel.exec.
The Kernel.exec call is from here https://github.com/rubygems/rubygems/blob/4968b9063e2d7d8261acc201c497197bc10b0f11/bundler/lib/bundler/self_manager.rb#L69-L97
and that acknowledges it loses all flags.
I think using Kernel.exec is fine on bundle exec as then flags are preserved and nothing is lost, and it's effectively transparent (except slightly slower to start Ruby again).
But on require "bundler/setup" I think Bundler should never do that.
It could warn or error that the "wrong" version of Bundler is loaded, and let the user retry.
That I think is the only way to properly preserve all that state and avoid lots of confusion.

BTW if I run:

$ ruby --yjit -Iharness-warmup benchmarks/activerecord/benchmark.rb     

Starting activerecord benchmark with:
PID: 613270
$LOAD_PATH
/home/eregon/code/yjit-bench/harness-warmup
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/site_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby/3.3.0/x86_64-linux
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/vendor_ruby
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0
/home/eregon/.rubies/ruby-3.3.5/lib/ruby/3.3.0/x86_64-linux
Dir.pwd: /home/eregon/code/yjit-bench
RUBY_DESCRIPTION: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [x86_64-linux]

ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [x86_64-linux]
Command: bundle check 2> /dev/null || bundle install
The Gemfile's dependencies are satisfied
/home/eregon/.rubies/ruby-3.3.5/bin/ruby: No such file or directory -- benchmarks/activerecord/benchmark.rb (LoadError)

Which illustrates the fact CWD has changed and that's a pretty cryptic error.

Did you try upgrading rubygems & bundler?

No, but https://github.com/rubygems/rubygems/blob/4968b9063e2d7d8261acc201c497197bc10b0f11/bundler/lib/bundler/self_manager.rb shows it's still present on master so I would expect that logic hasn't changed.

Start in bundler/lib/bundler/self_manager.rb, especially the self-management logic linked in the report, and reproduce the command using require "bundler/setup". Done means requiring Bundler no longer unexpectedly calls Kernel.exec or loses Ruby flags, the load path, or the current working directory.