bundler/inline activates an unexpected version

Describe the problem as clearly as you can

We're seeing some issues with bundler/inline activating an older bundler version picked up from the Gemfile.lock.

At best this is confusing, at worst we ran into some odd gem activation errors that I'm, so far, unable to reproduce in a concise way, but I can promise you they were awful to debug ;)

Did you try upgrading rubygems & bundler?

Yes.

Post steps to reproduce the problem

First, setup a non-inline Gemfile for the application:

cat <<EOF > Gemfile
source 'https://rubygems.org'

gem 'systemu'
EOF

and activate an older ruby version:

rbenv shell 2.7.8
bundle install

and check the bundle version:

$ bundle version
Bundler version 2.4.22 (2023-11-09 commit ec2089640)
$ grep -A1 BUNDLED Gemfile.lock
BUNDLED WITH
   2.4.22

looks good.

Now add our binstub helper, we want this using an independent version of ruby+bundler:

mkdir bin

cat <<EOF > bin/demo
#!/usr/bin/env ruby

require "bundler/inline"

gemfile do
  source 'https://rubygems.org'
  gem "uuidtools"
end

p Bundler::VERSION
EOF

chmod +x bin/demo

and test it

 $ ./bin/demo
"2.4.22"

okay, that's fine, but let's switch to a newer ruby, and 'accidentally' install the same older version of bundler:

rbenv shell 3.3.0
gem install bundler -v 2.4.22

which gives us

$ gem li | grep bundler
bundler (default: 2.5.3, 2.4.22)

this shouldn't matter if bundler/inline is independent, but

$ ./bin/demo
"2.4.22"

We unexpectedly activate the old version from the Gemfile.lock in the parent directory.

The current workaround for us is to set BUNDLE_GEMFILE to nothing:

$ BUNDLE_GEMFILE=/dev/null ./bin/demo
"2.5.3"

What were you expecting to happen?

bundler/inline to ignore any Gemfile.lock and use the most current version of bundler available.

What actually happened?

bundler/inline still used the 'BUNDLED WITH' version from the Gemfile.lock in the parent directory.

The current workaround for us is to set BUNDLE_GEMFILE

$ BUNDLE_GEMFILE=/dev/null ./bin/demo
"2.5.3"

I'm not clear how engrained the Gemfile.lock version activation is, but it seems less surprising to disable that for bundler/inline.

Start with the bundler/inline entry point and reproduce the reported commands with the parent Gemfile.lock and BUNDLE_GEMFILE workaround. Trace where the BUNDLED WITH version is activated, then identify the relevant existing tests or add coverage for inline execution; done means bundler/inline uses the current available Bundler version without the parent lockfile affecting it.