auto_install fails when bundler activates different versions of gems than what is in the bundle

Describe the problem as clearly as you can

With bundle config auto_install true, it's possible to have bundler gem dependencies activated during the auto_install that conflict with what is in a bundle. zlib is specific one I've run into, where my bundle had an older version locked (2.1.1), but installation activated 3.1.1, leading to this message:

You have already activated zlib 3.1.1, but your Gemfile requires zlib 2.1.1. Since zlib is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports zlib as a default gem.

Did you try upgrading rubygems & bundler?

Yes

Post steps to reproduce the problem

# Gemfile
# frozen_string_literal: true

source "https://rubygems.org"

gem "quickbooks-ruby"

Run:

$ bundle config auto_install true
$ bundle install
$ gem uninstall quickbooks-ruby
$ bundle console

Which command did you run?

bin/rubocop or any binstub that uses require 'bundler/setup' with

What were you expecting to happen?

bundler auto_install running to install gems shouldn't affect activating an application's bundle

What actually happened?

bundler activated zlib 3.1.1 during auto-install, and fails to activate zlib 2.1.1 for the application's bundle

If not included with the output of your command, run bundle env and paste the output below

Environment

Bundler       2.5.14
  Platforms   ruby, arm64-darwin-23
Ruby          3.3.2p78 (2024-05-30 revision e5a195edf62fe1bf7146a191da13fa1c4fecbd71) [arm64-darwin-23]
  Full Path   /Users/josh.nichols/.local/share/mise/installs/ruby/3.3.2/bin/ruby
  Config Dir  /Users/josh.nichols/.local/share/mise/installs/ruby/3.3.2/etc
RubyGems      3.5.9
  Gem Home    /Users/josh.nichols/.local/share/mise/installs/ruby/3.3.2/lib/ruby/gems/3.3.0
  Gem Path    /Users/josh.nichols/.gem/ruby/3.3.0:/Users/josh.nichols/.local/share/mise/installs/ruby/3.3.2/lib/ruby/gems/3.3.0
  User Home   /Users/josh.nichols
  User Path   /Users/josh.nichols/.gem/ruby/3.3.0
  Bin Dir     /Users/josh.nichols/.local/share/mise/installs/ruby/3.3.2/bin
Tools
  Git         2.45.2
  RVM         not installed
  rbenv       rbenv 1.2.0
  chruby

Bundler Build Metadata

Built At          2024-06-21
Git SHA           72920bc230
Released Version  true

Bundler settings

auto_install
  Set for your local app (/Users/josh.nichols/workspace/bundler-auto-install-already-activated-repo/.bundle/config): true
build.idn-ruby
  Set for the current user (/Users/josh.nichols/.bundle/config): "--with-idn-dir=/opt/homebrew/opt/libidn"
gem.changelog
  Set for the current user (/Users/josh.nichols/.bundle/config): false
gem.ci
  Set for the current user (/Users/josh.nichols/.bundle/config): false
gem.coc
  Set for the current user (/Users/josh.nichols/.bundle/config): false
gem.linter
  Set for the current user (/Users/josh.nichols/.bundle/config): "standard"
gem.mit
  Set for the current user (/Users/josh.nichols/.bundle/config): true
gem.test
  Set for the current user (/Users/josh.nichols/.bundle/config): false
silence_deprecations
  Set for the current user (/Users/josh.nichols/.bundle/config): true

Gemfile

Gemfile

# frozen_string_literal: true

source "https://rubygems.org"

gem "quickbooks-ruby"

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    activemodel (7.1.3.4)
      activesupport (= 7.1.3.4)
    activesupport (7.1.3.4)
      base64
      bigdecimal
      concurrent-ruby (~> 1.0, >= 1.0.2)
      connection_pool (>= 2.2.5)
      drb
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
      mutex_m
      tzinfo (~> 2.0)
    base64 (0.2.0)
    bigdecimal (3.1.8)
    concurrent-ruby (1.3.3)
    connection_pool (2.4.1)
    drb (2.2.1)
    faraday (2.9.2)
      faraday-net_http (>= 2.0, < 3.2)
    faraday-gzip (1.0.0)
      faraday (>= 1.0)
      zlib (~> 2.1)
    faraday-multipart (1.0.4)
      multipart-post (~> 2)
    faraday-net_http (3.1.0)
      net-http
    hashie (5.0.0)
    i18n (1.14.5)
      concurrent-ruby (~> 1.0)
    jwt (2.8.2)
      base64
    minitest (5.24.1)
    multi_xml (0.7.1)
      bigdecimal (~> 3.1)
    multipart-post (2.4.1)
    mutex_m (0.2.0)
    net-http (0.4.1)
      uri
    net-http-persistent (4.0.2)
      connection_pool (~> 2.2)
    nokogiri (1.16.6-aarch64-linux)
      racc (~> 1.4)
    nokogiri (1.16.6-arm-linux)
      racc (~> 1.4)
    nokogiri (1.16.6-arm64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.6-x86-linux)
      racc (~> 1.4)
    nokogiri (1.16.6-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.6-x86_64-linux)
      racc (~> 1.4)
    oauth2 (2.0.9)
      faraday (>= 0.17.3, < 3.0)
      jwt (>= 1.0, < 3.0)
      multi_xml (~> 0.5)
      rack (>= 1.2, < 4)
      snaky_hash (~> 2.0)
      version_gem (~> 1.1)
    quickbooks-ruby (2.0.3)
      activemodel (> 4.0)
      faraday (< 3.0)
      faraday-gzip (~> 1.0)
      faraday-multipart (~> 1.0)
      multipart-post
      net-http-persistent
      nokogiri
      oauth2 (< 3.0)
      roxml (~> 4.2)
    racc (1.8.0)
    rack (3.1.6)
    roxml (4.2.0)
      activesupport (>= 4.0)
      nokogiri (>= 1.3.3)
    snaky_hash (2.0.1)
      hashie
      version_gem (~> 1.1, >= 1.1.1)
    tzinfo (2.0.6)
      concurrent-ruby (~> 1.0)
    uri (0.13.0)
    version_gem (1.1.4)
    zlib (2.1.1)

PLATFORMS
  aarch64-linux
  arm-linux
  arm64-darwin
  x86-linux
  x86_64-darwin
  x86_64-linux

DEPENDENCIES
  quickbooks-ruby

BUNDLED WITH
   2.5.14

Reproduce the issue with bundle config auto_install true, bundle install, and bundle console, then compare the auto-install path with the binstub entry point using require 'bundler/setup'. Done means auto-install no longer activates a gem version that conflicts with the application's locked bundle, including the reported zlib case.