github / github/octocatalog-diff

puppetdb-ssl client cert auth used only for facts but not exported resources

Open
#194 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
252
Forks
95
PR merge metrics
No merged PRs in 30d

Description

When I run `octocatalog-diff` it succeeds gathering facts from puppetdb but later fails compiling the catalog while collecting exported resources indicating errors with OpenSSL.

- What did you do?

run octocatalog-diff against any node

- What happened?

could not retrieve stuff from puppetdb, failing with e.g. openssl errors

- What did you expect to happen?

`octocatalog-diff` output

- How can someone reproduce the problem?

1. set up puppet using external CA (in my case FreeIPA)
2. set up puppetdb to use this CA
3. get another certificate for running octocatalog-diff, signed by FreeIPA's CA
4. set up octocatalog-diff with puppetdb-ssl-ca, puppetdb-ssl-key and puppetdb-ssl-cert set according to docs
5. run octocatalog-diff

> Command used and debugging output

Command used: `octocatalog-diff -n puppet.example.com --debug`

Debugging output:

```
D, [2018-07-31T11:10:10.852241 #22274] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.3.1
D, [2018-07-31T11:10:10.853213 #22274] DEBUG -- : Command line arguments: ["--debug", "-n", "puppet.example.com"]
D, [2018-07-31T11:10:10.853564 #22274] DEBUG -- : Running on host portal (x86_64-linux-gnu)
D, [2018-07-31T11:10:10.853848 #22274] DEBUG -- : Compiling catalogs for puppet.example.com
D, [2018-07-31T11:10:10.854499 #22274] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for from-catalog
D, [2018-07-31T11:10:10.854846 #22274] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for to-catalog
D, [2018-07-31T11:10:10.855092 #22274] DEBUG -- : Initialized parallel task result array: size=2
D, [2018-07-31T11:10:10.856578 #22274] DEBUG -- : Launched pid=22276 for index=0
D, [2018-07-31T11:10:10.858113 #22274] DEBUG -- : Launched pid=22278 for index=1
D, [2018-07-31T11:10:10.859047 #22276] DEBUG -- : Begin build_catalog for production
D, [2018-07-31T11:10:10.859513 #22276] DEBUG -- : Setting up Puppet catalog build for production
D, [2018-07-31T11:10:10.859701 #22276] DEBUG -- : Catalog for production will be built with OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.859876 #22276] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.860127 #22276] DEBUG -- : Start retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.860333 #22276] DEBUG -- : Retrieving facts from PuppetDB
D, [2018-07-31T11:10:10.866278 #22278] DEBUG -- : Begin build_catalog for .
D, [2018-07-31T11:10:10.866789 #22278] DEBUG -- : Setting up Puppet catalog build for .
D, [2018-07-31T11:10:10.867026 #22278] DEBUG -- : Catalog for . will be built with OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867222 #22278] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867478 #22278] DEBUG -- : Start retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:10.867683 #22278] DEBUG -- : Retrieving facts from PuppetDB
D, [2018-07-31T11:10:11.011440 #22276] DEBUG -- : Success retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:11.013671 #22278] DEBUG -- : Success retrieving facts for puppet.example.com from OctocatalogDiff::Catalog::Computed
D, [2018-07-31T11:10:11.016847 #22278] DEBUG -- : Symlinked /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments/production -> /home/florian
/dev/puppet
D, [2018-07-31T11:10:11.017517 #22278] DEBUG -- : Installed puppetdb.conf file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/puppetdb.conf
D, [2018-07-31T11:10:11.019272 #22278] DEBUG -- : Installed routes.yaml file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/routes.yaml
D, [2018-07-31T11:10:11.022572 #22278] DEBUG -- : Installed hiera.yaml from /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments/production/
hiera.yaml to /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hiera.yaml
D, [2018-07-31T11:10:11.055529 #22276] DEBUG -- : ["Exit status: 0"]
D, [2018-07-31T11:10:11.055694 #22276] DEBUG -- : Success git archive /home/florian/dev/puppet:production
D, [2018-07-31T11:10:11.055731 #22276] DEBUG -- : Success git checkout /home/florian/dev/puppet:production -> /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20
180731-22276-4rk774
D, [2018-07-31T11:10:11.055787 #22276] DEBUG -- : Begin install bootstrap script in target directory
D, [2018-07-31T11:10:11.056114 #22276] DEBUG -- : Success: copied /home/florian/dev/puppet/bootstrap.sh to /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20180
731-22276-4rk774/bootstrap.sh
D, [2018-07-31T11:10:11.056180 #22276] DEBUG -- : Begin bootstrap with 'bootstrap.sh' in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-bootstrap-checkout-20180731-22276-4rk774
D, [2018-07-31T11:10:11.083511 #22278] DEBUG -- : Installed fact file at /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts/puppet.example.com.yaml
D, [2018-07-31T11:10:11.084687 #22278] DEBUG -- : Installed SSL client certificate in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/certs/
portal.pem
D, [2018-07-31T11:10:11.085269 #22278] DEBUG -- : Installed SSL client key in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/private_keys/p
ortal.pem
D, [2018-07-31T11:10:11.086038 #22278] DEBUG -- : Installed CA certificate in /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl/certs/ca.pem
D, [2018-07-31T11:10:12.959578 #22278] DEBUG -- : (to) Try 1 executing Puppet 4.10.12: /opt/puppetlabs/bin/puppet master --compile puppet.example.com --storeconfigs --
storeconfigs_backend=puppetdb --factpath=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts --facts_terminus=yaml --no-daemonize --no-ca
--color=false --config_version="/bin/echo catalogscript" --environment=production --hiera_config=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hi
era.yaml --environmentpath=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments --vardir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-201
80731-22278-2708v6/var --logdir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --ssldir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-2018073
1-22278-2708v6/var/ssl --confdir=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6
D, [2018-07-31T11:10:12.960353 #22278] DEBUG -- : ["Execute: /tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-scriptrunner20180731-22278-js3c2p/puppet.sh master --compile puppet.example.com --storeconfigs --storeconfigs_backend\\=puppetdb --factpath\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/yaml/facts --facts_te
rminus\\=yaml --no-daemonize --no-ca --color\\=false --config_version\\=\\\"/bin/echo catalogscript\\\" --environment\\=production --hiera_config\\=/tmp/ocd-ipc-20180731-
22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/hiera.yaml --environmentpath\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/environments --vardir
\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --logdir\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var --ssldir
\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6/var/ssl --confdir\\=/tmp/ocd-ipc-20180731-22274-p7r0rx/ocd-builddir-20180731-22278-2708v6"]

# skipped a few deprecation warnings from puppet modules here

[2018-07-31T11:10:31.080994 #22278] DEBUG -- : ["STDERR: Warning: Error connecting to puppet.example.com on 8081 at route /pdb/query/v4/resources?query=%5B%22and%22
%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D, error message received was 'SSL_connect SYSCALL returned=5 errno=0 state=unknown state'. Failing over to the next PuppetDB server_url in the 'server_urls' list"]
D, [2018-07-31T11:10:31.081028 #22278] DEBUG -- : ["STDERR: Error: Could not retrieve resources from the PuppetDB at puppet:8140: Failed to execute '/pdb/query/v4/resourc
es?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081057 #22278] DEBUG -- : ["STDERR: Error: Could not retrieve resources from the PuppetDB at puppet:8140: Failed to execute '/pdb/query/v4/resourc
es?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081089 #22278] DEBUG -- : ["STDERR: Error: Failed to compile catalog for node puppet.example.com: Could not retrieve resources from the PuppetD
B at puppet:8140: Failed to execute '/pdb/query/v4/resources?query=%5B%22and%22%2C%5B%22%3D%22%2C%22type%22%2C%22Sshkey%22%5D%2C%5B%22%3D%22%2C%22exported%22%2Ctrue%5D%2C%5B%22not%22%2C%5B%22%3D%22%2C%22certname%22%2C%22puppet.example.com%22%5D%5D%5D' on at least 1 of the following 'server_urls': https://puppet.example.com:8081 on node puppet.example.com"]
D, [2018-07-31T11:10:31.081124 #22278] DEBUG -- : ["Exit status: 30"]
W, [2018-07-31T11:10:31.081321 #22278] WARN -- : Puppet command failed: STDOUT:

# STDERR repeats

D, [2018-07-31T11:10:31.081532 #22278] DEBUG -- : Catalog for . failed with OctocatalogDiff::Catalog::Computed in 20.214317574 seconds
W, [2018-07-31T11:10:31.081666 #22278] WARN -- : Failed build_catalog for . validation: OctocatalogDiff::Errors::CatalogError Catalog failed:

# repeating deprecation warnings and error messages omitted

from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `call'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:39:in `validate'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:202:in `execute_task'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:119:in `block (2 levels) in run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `fork'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:117:in `block in run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `each_with_index'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:114:in `run_tasks_parallel'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/parallel.rb:94:in `run_tasks'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:92:in `build_catalog_parallelizer'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/util/catalogs.rb:29:in `catalogs'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1/catalog-diff.rb:34:in `catalog_diff'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/api/v1.rb:19:in `catalog_diff'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:119:in `cli'
from /var/lib/gems/2.3.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in `'
from /usr/local/bin/octocatalog-diff:23:in `load'
from /usr/local/bin/octocatalog-diff:23:in `'
```
> Platform and version information

- Your OS: Ubuntu 16.04 on all nodes
- Your Ruby version: `ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]` from Ubuntu 16.04
- Your version of Puppet: 4.10.12 from PC1 (open source)
- Your version of octocatalog-diff: 1.5.3 installed with `gem install octocatalog-diff`

> Do the tests pass from a clean checkout?

yes

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.