st2api service fails to restart after adding configuration for backend redis service
@lakshmi-kannan is already working on this.
Since Nov 22, 2016.
- Dominant language
- Python
- Stars
- 6.5k
- Forks
- 787
- PR merge metrics
- No merged PRs in 30d
Description
Using the Stackstorm 1.3 package-based install, I went in and configured redis in /etc/st2/st2.conf per the policies docs. This seemed to be fine initially but eventually something must have happened to cause the st2api service to restart and then (according to systemctl -l status st2api.service output) it looks like it got into some kind of restart loop.
I finally caught the offending output (this was pretty difficult to do and doesn't appear to be making its way into st2api.log), looks like the /opt/stackstorm/st2 Python virtualenv was missing the redis module:
st2api.service - StackStorm service st2api
Loaded: loaded (/usr/lib/systemd/system/st2api.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2016-04-21 10:42:01 EDT; 28s ago
Main PID: 26270 (gunicorn_pecan)
CGroup: /system.slice/st2api.service
\u251c\u250026270 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 127.0.0.1:9101 --threads 10 --workers 1 --graceful-timeout 10
\u2514\u250026522 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 127.0.0.1:9101 --threads 10 --workers 1 --graceful-timeout 10Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: File "/opt/stackstorm/st2/lib/python2.7/site-packages/stevedore/named.py", line 123, in _load_one_plugin
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: verify_requirements,
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: File "/opt/stackstorm/st2/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: plugin = ep.resolve()
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: File "/opt/stackstorm/st2/lib/python2.7/site-packages/pkg_resources/init.py", line 2208, in resolve
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: module = import(self.module_name, fromlist=['name'], level=0)
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: File "/opt/stackstorm/st2/lib/python2.7/site-packages/tooz/drivers/redis.py", line 27, in
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: import redis
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: ImportError: No module named redis
Apr 21 10:42:30 pxops-st2 gunicorn_pecan[26270]: [2016-04-21 10:42:30 +0000] [26522] [INFO] Worker exiting (pid: 26522)
My fix was to install the redis module into the virtualenv, ie. sudo su; source /opt/stackstorm/st2/bin/activate; pip install redis
After this systemctl restart st2api appeared to work and everything was happy again.
Not sure if this is a missing dependency in packaging or a setup script or just needs to be noted in the docs. Let me know if you need any more info about my setup.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.