StackStorm / StackStorm/st2

Failed to connect stackstorm in k8s to MongoDB atlas

Open
#5,518 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

Receiving that err for mongo atlas connection with stackstorm in kubernetes:
2021-12-27 12:39:16,014 INFO [-] Patching sys.stdout and re-opening it with utf-8 encoding (originally opened with UTF-8 encoding)...
2021-12-27 12:39:16,016 INFO [-] Using Python: 3.6.9 (/opt/stackstorm/st2/bin/python)
2021-12-27 12:39:16,016 INFO [-] Using fs encoding: utf-8, default encoding: utf-8, locale: en_US.UTF-8, LANG env variable: en_US.UTF-8, PYTHONIOENCODING env variable: notset
2021-12-27 12:39:16,017 INFO [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf
2021-12-27 12:39:16,017 INFO [-] Using logging config: /etc/st2/logging.api.gunicorn.conf
2021-12-27 12:39:16,065 INFO [-] Connecting to database "st2" @ "USER.azure.mongodb.net:27017,USER.azure.mongodb.net:27017USERazure.mongodb.net:27017 (replica set)" as user "st2-t".
2021-12-27 12:39:19,149 ERROR [-] Failed to connect to database "st2" @ "USER.azure.mongodb.net:27017,USER.azure.mongodb.net:27017 (replica set)" as user "st2-t": connection closed,connection closed,connection closed
2021-12-27 12:39:19,150 ERROR [-] (PID=1) ST2 API quit due to exception.
Traceback (most recent call last):
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/cmd/api.py", line 85, in main
_setup()
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/cmd/api.py", line 59, in _setup
service_registry=True, capabilities=capabilities)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/service_setup.py", line 226, in setup
db_setup()
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/database_setup.py", line 57, in db_setup
connection = db_init.db_setup_with_retry(**db_cfg)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/persistence/db_init.py", line 76, in db_setup_with_retry
ssl_match_hostname=ssl_match_hostname)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/persistence/db_init.py", line 59, in db_func_with_retry
return retrying_obj.call(db_func, *args, **kwargs)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 206, in call
return attempt.get(self._wrap_exception)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/opt/stackstorm/st2/lib/python3.6/site-packages/six.py", line 696, in reraise
raise value
File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 170, in db_setup
ssl_match_hostname=ssl_match_hostname)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 152, in _db_connect
raise e
File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 145, in _db_connect
connection.admin.command('ismaster')
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/database.py", line 730, in command
read_preference, session) as (sock_info, slave_ok):
File "/usr/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1298, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1253, in _select_server
server = topology.select_server(server_selector)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 235, in select_server
address))
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 193, in select_servers
selector, server_timeout, address)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 209, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

My conf:
[database]
host = "mongodb+srv://st2-t:f45ew2.mongodb.net/st2-t?retryWrites=true&w=majority"
username = st2-t
password = f45ew2
db_name = st2-t
port = 27017
ssl = False

env: stackstorm 3.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the [database] configuration and the traceback entry points in st2common/persistence/db_init.py and st2common/models/db/init.py, especially _db_connect. Reproduce the MongoDB Atlas connection from the Kubernetes deployment and determine whether the failure is configuration or connection handling; done means a confirmed fix or a documented, reproducible cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, mongodb, python
Domain
databases, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.