ruby / ruby/rubygems

`bundle install --local` ignores installed gems as long as their copies are available in cache

Open
#9,380 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bundler
Dominant language
Ruby
Stars
4k
Forks
1.9k
Avg merge
1d 2h
Merged PRs (30d)
81

Description

So I have a RoR 8.1 application and can do successfully:

$ ~/bin/bundle install --gemfile /builddir/tr/Gemfile --local
Resolving dependencies...
Bundle complete! 14 Gemfile dependencies, 91 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
1 installed gem you directly depend on is looking for funding.
  Run `bundle fund` for details

When I remove the --local option, it fails with multiple errors:

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

extconf failed, exit code 1

Gem files will remain installed in /builddir/.local/share/gem/ruby/gems/erb-6.0.1 for inspection.
Results logged to /builddir/.local/share/gem/ruby/extensions/x86_64-linux/4.0.0/erb-6.0.1/gem_make.out

  /usr/share/rubygems/rubygems/ext/builder.rb:139:in 'Gem::Ext::Builder.run'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:30:in 'Gem::Ext::ExtConfBuilder.build'
  /usr/share/rubygems/rubygems/ext/builder.rb:218:in 'Gem::Ext::Builder#build_extension'
  /usr/share/rubygems/rubygems/ext/builder.rb:250:in 'block in Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Array#each'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/installer.rb:809:in 'Gem::Installer#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:112:in 'Bundler::RubyGemsGemInstaller#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:30:in 'Bundler::RubyGemsGemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:217:in 'block in Bundler::Source::Rubygems#install'
  /usr/share/rubygems/rubygems.rb:1052:in 'Gem.time'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:216:in 'Bundler::Source::Rubygems#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:54:in 'Bundler::GemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:17:in 'Bundler::GemInstaller#install_from_spec'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:133:in 'Bundler::ParallelInstaller#do_install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:124:in 'block in Bundler::ParallelInstaller#worker_pool'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:62:in 'Bundler::Worker#apply_func'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:57:in 'block in Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Kernel#loop'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:90:in 'block (2 levels) in Bundler::Worker#create_threads'

An error occurred while installing erb (6.0.1), and Bundler cannot continue.

In Gemfile:
  rails was resolved to 8.1.2, which depends on
    actiontext was resolved to 8.1.2, which depends on
      action_text-trix was resolved to 2.1.16, which depends on
        railties was resolved to 8.1.2, which depends on
          irb was resolved to 1.16.0, which depends on
            rdoc was resolved to 7.0.3, which depends on
              erb


Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /builddir/.local/share/gem/ruby/gems/date-3.5.1/ext/date
/usr/bin/ruby -I/usr/share/rubygems extconf.rb
mkmf.rb can't find header files for ruby at /usr/share/include/ruby.h

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

extconf failed, exit code 1

Gem files will remain installed in /builddir/.local/share/gem/ruby/gems/date-3.5.1 for inspection.
Results logged to /builddir/.local/share/gem/ruby/extensions/x86_64-linux/4.0.0/date-3.5.1/gem_make.out

  /usr/share/rubygems/rubygems/ext/builder.rb:139:in 'Gem::Ext::Builder.run'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:30:in 'Gem::Ext::ExtConfBuilder.build'
  /usr/share/rubygems/rubygems/ext/builder.rb:218:in 'Gem::Ext::Builder#build_extension'
  /usr/share/rubygems/rubygems/ext/builder.rb:250:in 'block in Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Array#each'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/installer.rb:809:in 'Gem::Installer#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:112:in 'Bundler::RubyGemsGemInstaller#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:30:in 'Bundler::RubyGemsGemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:217:in 'block in Bundler::Source::Rubygems#install'
  /usr/share/rubygems/rubygems.rb:1052:in 'Gem.time'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:216:in 'Bundler::Source::Rubygems#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:54:in 'Bundler::GemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:17:in 'Bundler::GemInstaller#install_from_spec'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:133:in 'Bundler::ParallelInstaller#do_install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:124:in 'block in Bundler::ParallelInstaller#worker_pool'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:62:in 'Bundler::Worker#apply_func'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:57:in 'block in Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Kernel#loop'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:90:in 'block (2 levels) in Bundler::Worker#create_threads'

An error occurred while installing date (3.5.1), and Bundler cannot continue.

In Gemfile:
  rails was resolved to 8.1.2, which depends on
    actiontext was resolved to 8.1.2, which depends on
      action_text-trix was resolved to 2.1.16, which depends on
        railties was resolved to 8.1.2, which depends on
          irb was resolved to 1.16.0, which depends on
            rdoc was resolved to 7.0.3, which depends on
              psych was resolved to 5.3.1, which depends on
                date


Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /builddir/.local/share/gem/ruby/gems/stringio-3.2.0/ext/stringio
/usr/bin/ruby -I/usr/share/rubygems extconf.rb
mkmf.rb can't find header files for ruby at /usr/share/include/ruby.h

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

extconf failed, exit code 1

Gem files will remain installed in /builddir/.local/share/gem/ruby/gems/stringio-3.2.0 for inspection.
Results logged to /builddir/.local/share/gem/ruby/extensions/x86_64-linux/4.0.0/stringio-3.2.0/gem_make.out

  /usr/share/rubygems/rubygems/ext/builder.rb:139:in 'Gem::Ext::Builder.run'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:30:in 'Gem::Ext::ExtConfBuilder.build'
  /usr/share/rubygems/rubygems/ext/builder.rb:218:in 'Gem::Ext::Builder#build_extension'
  /usr/share/rubygems/rubygems/ext/builder.rb:250:in 'block in Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Array#each'
  /usr/share/rubygems/rubygems/ext/builder.rb:249:in 'Gem::Ext::Builder#build_extensions'
  /usr/share/rubygems/rubygems/installer.rb:809:in 'Gem::Installer#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:112:in 'Bundler::RubyGemsGemInstaller#build_extensions'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/rubygems_gem_installer.rb:30:in 'Bundler::RubyGemsGemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:217:in 'block in Bundler::Source::Rubygems#install'
  /usr/share/rubygems/rubygems.rb:1052:in 'Gem.time'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/source/rubygems.rb:216:in 'Bundler::Source::Rubygems#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:54:in 'Bundler::GemInstaller#install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/gem_installer.rb:17:in 'Bundler::GemInstaller#install_from_spec'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:133:in 'Bundler::ParallelInstaller#do_install'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/installer/parallel_installer.rb:124:in 'block in Bundler::ParallelInstaller#worker_pool'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:62:in 'Bundler::Worker#apply_func'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:57:in 'block in Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Kernel#loop'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:54:in 'Bundler::Worker#process_queue'
  /builddir/.local/share/gem/ruby/gems/bundler-4.0.7/lib/bundler/worker.rb:90:in 'block (2 levels) in Bundler::Worker#create_threads'

An error occurred while installing stringio (3.2.0), and Bundler cannot continue.

In Gemfile:
  rails was resolved to 8.1.2, which depends on
    actiontext was resolved to 8.1.2, which depends on
      action_text-trix was resolved to 2.1.16, which depends on
        railties was resolved to 8.1.2, which depends on
          irb was resolved to 1.16.0, which depends on
            rdoc was resolved to 7.0.3, which depends on
              psych was resolved to 5.3.1, which depends on
                stringio

So far so good. But if I put the --local option back, it keeps failing with the previous error:

$ ~/bin/bundle install --gemfile /builddir/tr/Gemfile --local
Resolving dependencies...
Installing stringio 3.2.0 with native extensions
Installing erb 6.0.1 with native extensions
Installing date 3.5.1 with native extensions
Installing prism 1.8.0 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /builddir/.local/share/gem/ruby/gems/prism-1.8.0/ext/prism

... snip ...

An error occurred while installing stringio (3.2.0), and Bundler cannot continue.

In Gemfile:
  rails was resolved to 8.1.2, which depends on
    actiontext was resolved to 8.1.2, which depends on
      action_text-trix was resolved to 2.1.16, which depends on
        railties was resolved to 8.1.2, which depends on
          irb was resolved to 1.16.0, which depends on
            rdoc was resolved to 7.0.3, which depends on
              psych was resolved to 5.3.1, which depends on
                stringio

and it does so as long as the failing gems are in cache. IOW this helps:

$ rm ~/.local/share/gem/ruby/cache/{date,erb,prism,stringio}*.gem 
$ ~/bin/bundle install --gemfile /builddir/tr/Gemfile --local
Resolving dependencies...
Bundle complete! 14 Gemfile dependencies, 91 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
1 installed gem you directly depend on is looking for funding.
  Run `bundle fund` for details

I don't think that this is expected behavior. The cache is just cache. Availability of .gem file in it should not influence the result. It should only help with efficiency which is not the case.

$ gem info bundler

*** LOCAL GEMS ***

bundler (4.0.7)
    Authors: André Arko, Samuel Giddins, Colby Swandale, Hiroshi
    Shibata, David Rodríguez, Grey Baker, Stephanie Morillo, Chris
    Morris, James Wen, Tim Moore, André Medeiros, Jessica Lynn Suttles,
    Terence Lee, Carl Lerche, Yehuda Katz
    Homepage: https://bundler.io
    License: MIT
    Installed at: /builddir/.local/share/gem/ruby

    The best way to manage your application's dependencies

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the behavior with bundle install --local while the affected gem packages remain in the cache, then compare it after removing those cached gems. Start with Bundler::Source::Rubygems#install and Bundler::RubyGemsGemInstaller#build_extensions from the reported trace. Done means cached copies do not cause already installed gems to be rebuilt and the local install completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.