github / github/octocatalog-diff
puppetdb-ssl client cert auth used only for facts but not exported resources
- Dominant language
- Ruby
- Stars
- 252
- Forks
- 95
- PR merge metrics
- No merged PRs in 30d
Description
When I run `octocatalog-diff` it succeeds gathering facts from puppetdb but later fails compiling the catalog while collecting exported resources indicating errors with OpenSSL.
- What did you do?
run octocatalog-diff against any node
- What happened?
could not retrieve stuff from puppetdb, failing with e.g. openssl errors
- What did you expect to happen?
`octocatalog-diff` output
- How can someone reproduce the problem?
1. set up puppet using external CA (in my case FreeIPA)
2. set up puppetdb to use this CA
3. get another certificate for running octocatalog-diff, signed by FreeIPA's CA
4. set up octocatalog-diff with puppetdb-ssl-ca, puppetdb-ssl-key and puppetdb-ssl-cert set according to docs
5. run octocatalog-diff
> Command used and debugging output
Command used: `octocatalog-diff -n puppet.example.com --debug`
Debugging output:
```
D, [2018-07-31T11:10:10.852241 #22274] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.3.1
D, [2018-07-31T11:10:10.853213 #22274] DEBUG -- : Command line arguments: ["--debug", "-n", "puppet.example.com"]
D, [2018-07-31T11:10:10.853564 #22274] DEBUG -- : Running on host portal (x86_64-linux-gnu)
D, [2018-07-31T11:10:10.853848 #22274] DEBUG -- : Compiling catalogs for puppet.example.com
D, [2018-07-31T11:10:10.854499 #22274] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for from-catalog
D, [2018-07-31T11:10:10.854846 #22274] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for to-catalog
D, [2018-07-31T11:10:10.855092 #22274] DEBUG -- : Initialized parallel task result array: size=2
D, [2018-07-31T11:10:10.856578 #22274] DEBUG -- : Launched pid=22276 for index=0
D, [2018-07-31T11:10:10.858113 #22274] DEBUG -- : Launched pid=22278 for index=1
D, [2018-07-31T11:10:10.859047 #22276] DEBUG -- : Begin build_catalog for production
D, [2018-07-31T11:10:10.859513 #22276] DEBUG -- : Setting up Puppet catalog build for production
D, [2018-07-31T11:10:10.859701 #22276] DEBUG -- : Catalog for production will be built with OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.859876 #22276] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.860127 #22276] DEBUG -- : Start retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.860333 #22276] DEBUG -- : Retrieving facts from PuppetDB
D, [2018-07-31T11:10:10.866278 #22278] DEBUG -- : Begin build_catalog for .
D, [2018-07-31T11:10:10.866789 #22278] DEBUG -- : Setting up Puppet catalog build for .
D, [2018-07-31T11:10:10.867026 #22278] DEBUG -- : Catalog for . will be built with OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867222 #22278] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867478 #22278] DEBUG -- : Start retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867683 #22278] DEBUG -- : Retrieving facts from PuppetDB
D, [2018-07-31T11:10:11.011440 #22276] DEBUG -- : Success retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:11.013671 #22278] DEBUG -- : Success retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:11.016847 #22278] DEBUG -- : Symlinked /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments/production -> /home/florian
/dev/puppet
D, [2018-07-31T11:10:11.017517 #22278] DEBUG -- : Installed puppetdb.conf file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/puppetdb.conf
D, [2018-07-31T11:10:11.019272 #22278] DEBUG -- : Installed routes.yaml file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/routes.yaml
D, [2018-07-31T11:10:11.022572 #22278] DEBUG -- : Installed hiera.yaml from /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments/production/
hiera.yaml to /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hiera.yaml
D, [2018-07-31T11:10:11.055529 #22276] DEBUG -- : ["Exit status: 0"]
D, [2018-07-31T11:10:11.055694 #22276] DEBUG -- : Success git archive /home/florian/dev/puppet:production
D, [2018-07-31T11:10:11.055731 #22276] DEBUG -- : Success git checkout /home/florian/dev/puppet:production -> /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20
180731-22276-4rk774
D, [2018-07-31T11:10:11.055787 #22276] DEBUG -- : Begin install bootstrap script in target directory
D, [2018-07-31T11:10:11.056114 #22276] DEBUG -- : Success: copied /home/florian/dev/puppet/bootstrap.sh to /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20180
731-22276-4rk774/bootstrap.sh
D, [2018-07-31T11:10:11.056180 #22276] DEBUG -- : Begin bootstrap with 'bootstrap.sh' in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20180731-22276-4rk774
D, [2018-07-31T11:10:11.083511 #22278] DEBUG -- : Installed fact file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts/puppet.example.com.yaml
D, [2018-07-31T11:10:11.084687 #22278] DEBUG -- : Installed SSL client certificate in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/certs/
portal.pem
D, [2018-07-31T11:10:11.085269 #22278] DEBUG -- : Installed SSL client key in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/private_keys/p
ortal.pem
D, [2018-07-31T11:10:11.086038 #22278] DEBUG -- : Installed CA certificate in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/certs/ca.pem
D, [2018-07-31T11:10:12.959578 #22278] DEBUG -- : (to) Try 1 executing Puppet 4.10.12: /opt/puppetlabs/bin/puppet master --compile puppet.example.com --storeconfigs --
storeconfigs_backend=puppetdb --factpath=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts --facts_terminus=yaml --no-daemonize --no-ca
--color=false --config_version="/bin/echo catalogscript" --environment=production --hiera_config=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hi
era.yaml --environmentpath=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments --vardir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-201
80731-22278-2708v6/var --logdir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --ssldir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-2018073
1-22278-2708v6/var/ssl --confdir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6
D, [2018-07-31T11:10:12.960353 #22278] DEBUG -- : ["Execute: /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-scriptrunner20180731-22278-js3c2p/puppet.sh master --compile puppet.example.com --storeconfigs --storeconfigs_backend\\=puppetdb --factpath\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts --facts_te
rminus\\=yaml --no-daemonize --no-ca --color\\=false --config_version\\=\\\"/bin/echo catalogscript\\\" --environment\\=production --hiera_config\\=/tmp/ocd-ipc-20180731-
22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hiera.yaml --environmentpath\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments --vardir
\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --logdir\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --ssldir
\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl --confdir\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6"]
# skipped a few deprecation warnings from puppet modules here
[2018-07-31T11:10:31.080994 #22278] DEBUG -- : ["STDERR: Warning: Error connecting to puppet.example.com on 8081 at route /pdb/query/v4/resources?query=%5B%22and%22
%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D, error message received was 'SSL_connect SYSCALL returned=5 errno=0 state=unknown state'. Failing over to the next PuppetDB server_url in the 'server_urls' list"]
D, [2018-07-31T11:10:31.081028 #22278] DEBUG -- : ["STDERR: Error: Could not retrieve resources from the PuppetDB at puppet:8140: Failed to execute '/pdb/query/v4/resourc
es?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081057 #22278] DEBUG -- : ["STDERR: Error: Could not retrieve resources from the PuppetDB at puppet:8140: Failed to execute '/pdb/query/v4/resourc
es?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081089 #22278] DEBUG -- : ["STDERR: Error: Failed to compile catalog for node puppet.example.com: Could not retrieve resources from the PuppetD
B at puppet:8140: Failed to execute '/pdb/query/v4/resources?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081124 #22278] DEBUG -- : ["Exit status: 30"]
W, [2018-07-31T11:10:31.081321 #22278] WARN -- : Puppet command failed: STDOUT:
# STDERR repeats
D, [2018-07-31T11:10:31.081532 #22278] DEBUG -- : Catalog for . failed with OctocatalogDiff::Catalog::Computed in 20.214317574 seconds
W, [2018-07-31T11:10:31.081666 #22278] WARN -- : Failed build_catalog for . validation: OctocatalogDiff::Errors::CatalogError Catalog failed:
# repeating deprecation warnings and error messages omitted
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `call'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `validate'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:202:in `execute_task'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:119:in `block (2 levels) in run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `fork'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `block in run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each_with_index'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:94:in `run_tasks'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:92:in `build_catalog_parallelizer'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:29:in `catalogs'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1/catalog-diff.rb:34:in `catalog_diff'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1.rb:19:in `catalog_diff'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:119:in `cli'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in `'
from /usr/local/bin/octocatalog-diff:23:in `load'
from /usr/local/bin/octocatalog-diff:23:in `'
```
> Platform and version information
- Your OS: Ubuntu 16.04 on all nodes
- Your Ruby version: `ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]` from Ubuntu 16.04
- Your version of Puppet: 4.10.12 from PC1 (open source)
- Your version of octocatalog-diff: 1.5.3 installed with `gem install octocatalog-diff`
> Do the tests pass from a clean checkout?
yes
Contributor guide
Assessment
This issue has not been assessed yet.