reload_modules: True does not reload module dependencies
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Hi,
I would like to use the swiftexecution module to install an application.
For the swift execution module to be enabled, one need to install the python-swiftclient library first.
$ salt --versions-report
Salt: 2015.5.2-120-g1650233
Python: 2.7.6 (default, Mar 22 2014, 22:59:56)
Jinja2: 2.7.2
M2Crypto: 0.21.1
msgpack-python: 0.4.6
msgpack-pure: Not Installed
pycrypto: 2.6.1
libnacl: Not Installed
PyYAML: 3.10
ioflo: Not Installed
PyZMQ: 14.0.1
RAET: Not Installed
ZMQ: 4.0.4
Mako: Not Installed
Tornado: Not Installed
Here is a simple SLS file:
python-swiftclient:
pip.installed:
- reload_modules: True
python-keystoneclient:
pip.installed:
- reload_modules: True
python-dev:
pkg.installed
download_application:
module.run:
- name: swift.get
- cont: test_container
- path: test_app
- local_file: /tmp/test_app
- require:
- pip: python-keystoneclient
- pip: python-swiftclient
The execution of this SLS file fails at the download_application step because it says that the swift module is not present:
Changes:
----------
ID: download_application
Function: module.run
Name: swift.get
Result: False
Comment: Module function swift.get is not available
Started: 12:42:28.767301
Duration: 200.984 ms
Changes:
In order to make it work, I need to restart the minion to be aware of the new python-swiftclient library being installed, and re-run the SLS file.
By looking at the source code, we can see that the python reload() function is called for the modules/swift.py module, so the module is effectively reloaded.
BUT, its dependencies are not (this is not the behaviour of the reload() function)
We can see at https://github.com/saltstack/salt/blob/develop/salt/modules/swift.py#L46 that swift.py imports salt.utils.openstack.swift that is "evaluated" once, on first import (re-importing a library does not mean the library is re-evaluated)
So at https://github.com/saltstack/salt/blob/develop/salt/utils/openstack/swift.py#L22 the following code is only executed once and the variable HAS_SWIFT is still set to False even after the swift.py module is reloaded because reload() does not reload python dependencies.
# Import Swift client libs
HAS_SWIFT = False
try:
from swiftclient import client
HAS_SWIFT = True
except ImportError:
pass
Other modules like docker.io are not affected by this problem because they directly import third-party libraries and upon reloading, they retry to import these libraries. So installing docker-py with reload_modules: True enables the docker.io execution module without restarting the minion.
Adding a reload(suos) call in the virtual() function fixes the problem but this is not a clean solution.
Questions:
- Do I install the third party library the right way ?
- Should I expect that the installation of a third party library is taken into account without restarting the minion ?
Cheers.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with salt/modules/swift.py and salt/utils/openstack/swift.py, then reproduce the SLS sequence using pip.installed with reload_modules: True and the swift.get module call. Compare how dependency availability is evaluated after installation and define completion as making the Swift execution module recognize the installed client without restarting the minion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100