pkg.installed with sources doesn't work behind proxy
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue/Question
When a minion runs behind a proxy, all pkg.installed states succeed, except those that define sources.
Setup
install_osquery:
pkg.installed:
- sources:
- osquery: https://pkg.osquery.io/rpm/osquery-2.10.2-1.linux.x86_64.rpm
Steps to Reproduce Issue
Invoke the above state on the minion behind the proxy,
Logs:
[INFO ] Running state [install_osquery] at time 15:50:07.614530
[INFO ] Executing state pkg.installed for [install_osquery]
[DEBUG ] Requesting URL https://pkg.osquery.io/rpm/osquery-2.10.2-1.linux.x86_64.rpm using GET method
[DEBUG ] Using backend: requests
[ERROR ] An exception occurred in this state: Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/state.py", line 1905, in call
**cdata['kwargs'])
File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1830, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/states/pkg.py", line 1548, in installed
**kwargs)
File "/usr/lib/python2.7/site-packages/salt/states/pkg.py", line 716, in _find_install_targets
cached_path = __salt__['cp.cache_file'](val, saltenv=kwargs['saltenv'])
File "/usr/lib/python2.7/site-packages/salt/modules/cp.py", line 500, in cache_file
result = _client().cache_file(path, saltenv, source_hash=source_hash)
File "/usr/lib/python2.7/site-packages/salt/fileclient.py", line 193, in cache_file
path, '', True, saltenv, cachedir=cachedir, source_hash=source_hash)
File "/usr/lib/python2.7/site-packages/salt/fileclient.py", line 734, in get_url
**get_kwargs
File "/usr/lib/python2.7/site-packages/salt/utils/http.py", line 349, in query
method, url, params=params, data=data, **req_kwargs
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(110, 'Connection timed out'))
Versions Report
$ salt --versions-report
Salt Version:
Salt: 2018.3.2
Dependency Versions:
cffi: 1.10.0
cherrypy: unknown
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.8.1
libgit2: 0.26.0
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: 2.18
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.26.0
Python: 2.7.14 (default, Jan 31 2018, 02:12:13)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 14.5.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
System Versions:
dist: centos 6.10 Final
locale: UTF-8
machine: x86_64
release: 2.6.32-696.16.1.el6.x86_64
system: Linux
version: CentOS 6.10 Final
I export the https_proxy, http_proxy and no_proxy env vars in file /etc/profile.d/proxy.sh. Also, using the requests backend.
Observing the logs of our proxy server, other requests that are going out to the internet appear normally. The above, problematic request never reaches our proxy server.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with salt/states/pkg.py in _find_install_targets, then follow cp.py cache_file into salt/fileclient.py get_url and salt/utils/http.py query. Reproduce the pkg.installed state with the osquery source behind the configured proxy and compare its request path with the logs. Done means the source package request uses the proxy and the state succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100