hashicorp / hashicorp/vagrant-spec

It's unclear how to run tests against 3rd party providers

Open
#2 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
80
Forks
26
PR merge metrics
No merged PRs in 30d

Description

I'm trying to run the vagrant-spec against the vagrant-kvm plugin. I'm running in an issue where it looks like the plugin is not installed.

I think what's going on is the vagrant-spec shells out to a different process the knows nothing of vagrant-kvm. I also want to test against my local version of the plugin, not the latest release.

Is there a way of achieving this currently?

My vagrant-spec.config.rb file:

``` ruby
require 'vagrant-kvm'
require "vagrant-spec/acceptance"

Vagrant::Spec::Acceptance.configure do |c|
c.provider "kvm", box: "/home/vagrant/vagrant-kvm/spec/precise-server-cloudimg-amd64-vagrant-disk1.box"
end
```

full error message:

```
1) provider/kvm/basic it should behave like provider/basic after an up can manage machine lifecycle
Failure/Error: assert_execute("vagrant", "up", "--provider=#{provider}")
expected command to exit with 0 but got exit code: 1

stdout: You appear to be running Vagrant in a Bundler environment. Because
Vagrant should be run within installers (outside of Bundler), Vagrant
will assume that you're developing plugins and will change its behavior
in certain ways to better assist plugin development.

stderr: The provider 'kvm' could not be found, but was requested to
back the machine 'default'. Please use a provider that exists.
Shared Example Group: "provider/basic" called from
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-expectations-2.14.4/lib/rspec/expectations/fail_with.rb:32:in `fail_with'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-expectations-2.14.4/lib/rspec/expectations/handler.rb:36:in `handle_matcher'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-expectations-2.14.4/lib/rspec/expectations/expectation_target.rb:34:in `to'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/lib/vagrant-spec/acceptance/rspec/context.rb:55:in `assert_execute'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/acceptance/provider/basic_spec.rb:50:in `block (3 levels) in '
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:237:in `instance_eval'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:237:in `instance_eval'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/hooks.rb:21:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/hooks.rb:85:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/hooks.rb:85:in `each'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/hooks.rb:85:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/hooks.rb:446:in `run_hook'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:345:in `run_before_each_hooks'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:294:in `run_before_each'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:113:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:254:in `with_around_each_hooks'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example.rb:111:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:390:in `block in run_examples'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:386:in `map'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:386:in `run_examples'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:371:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `map'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `map'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/example_group.rb:372:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/command_line.rb:28:in `map'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/command_line.rb:28:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/reporter.rb:58:in `report'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/command_line.rb:25:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/rspec-core-2.14.7/lib/rspec/core/runner.rb:80:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/lib/vagrant-spec/acceptance/runner.rb:43:in `block in run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/lib/vagrant-spec/acceptance/runner.rb:76:in `with_world'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/lib/vagrant-spec/acceptance/runner.rb:28:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/lib/vagrant-spec/cli.rb:34:in `test'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/thor-0.18.1/lib/thor/command.rb:27:in `run'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/thor-0.18.1/lib/thor/invocation.rb:120:in `invoke_command'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/thor-0.18.1/lib/thor.rb:363:in `dispatch'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/gems/thor-0.18.1/lib/thor/base.rb:439:in `start'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bundler/gems/vagrant-spec-a2bae8505c56/bin/vagrant-spec:7:in `'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bin/vagrant-spec:23:in `load'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bin/vagrant-spec:23:in `'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bin/ruby_executable_hooks:15:in `eval'
# /home/vcontrol/.rvm/gems/ruby-2.0.0-p353@vagrant-kvm/bin/ruby_executable_hooks:15:in `'

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the vagrant-spec.config.rb example and trace the acceptance runner through lib/vagrant-spec/acceptance/runner.rb and lib/vagrant-spec/acceptance/rspec/context.rb, where the failing vagrant command is assembled. Use acceptance/provider/basic_spec.rb and the reported provider-not-found output to understand the current boundary. Done means acceptance tests can use a third-party provider and a local plugin version as requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.