Stackstorm deployment failed on FIPS enabled OCP env

SUMMARY

Deploy Stackstorm failed on FIPS enabled OCP env.

This is follow up of the issue https://github.com/StackStorm/stackstorm-ha/issues/174, it seems related to StackStorm core implementation and I'm suggested to open an issue here.

STACKSTORM VERSION

3.2.0

OS, environment, install method

Installed with stackstorm HA on OCP env with FIPS enabled.

Steps to reproduce the problem

Deploy Stackstorm failed on FIPS enabled OCP env as below

chatops-st2actionrunner-d77546c98-knq4x        0/1     Error              1          17s
chatops-st2actionrunner-d77546c98-kxz7q        0/1     Error              1          17s
chatops-st2actionrunner-d77546c98-mszv9        0/1     Error              1          17s
chatops-st2actionrunner-d77546c98-nrqbc        0/1     Error              1          17s
chatops-st2actionrunner-d77546c98-zb2xf        0/1     CrashLoopBackOff   1          17s
chatops-st2api-555d6555d-9jfb2                 0/1     CrashLoopBackOff   1          17s
chatops-st2api-555d6555d-pqn9z                 0/1     CrashLoopBackOff   1          17s
chatops-st2auth-797688d59d-pvv82               0/1     CrashLoopBackOff   1          17s
chatops-st2auth-797688d59d-z6mz6               0/1     CrashLoopBackOff   1          17s
chatops-st2client-575464b649-n6hzn             1/1     Terminating        0          11m
chatops-st2garbagecollector-86b776d776-kppwr   0/1     CrashLoopBackOff   1          17s
chatops-st2notifier-7c65496c94-5cj5b           0/1     CrashLoopBackOff   1          17s
chatops-st2notifier-7c65496c94-ffx24           0/1     CrashLoopBackOff   1          17s
chatops-st2rulesengine-59fdb87778-ksh8f        0/1     CrashLoopBackOff   1          17s
chatops-st2rulesengine-59fdb87778-wmqtz        0/1     ImagePullBackOff   0          17s
chatops-st2scheduler-7c8cfddb7-mm6fx           0/1     CrashLoopBackOff   1          17s
chatops-st2scheduler-7c8cfddb7-pmmft           0/1     CrashLoopBackOff   1          17s
chatops-st2sensorcontainer-787fb46b6f-zsmdw    0/1     CrashLoopBackOff   1          17s
chatops-st2stream-744758965f-mz49j             0/1     CrashLoopBackOff   1          17s
chatops-st2stream-744758965f-x5259             0/1     CrashLoopBackOff   1          17s
chatops-st2timersengine-cb5c5b4b5-5j2b7        0/1     ImagePullBackOff   0          17s
chatops-st2workflowengine-7ffdbfccdd-6jl4f     0/1     Error              1          17s
chatops-st2workflowengine-7ffdbfccdd-bhcx2     0/1     CrashLoopBackOff   1          17s

failed pod has following error related to FIPS.

Traceback (most recent call last):
  File "/opt/stackstorm/st2/bin/st2auth", line 19, in <module>
    from st2auth.cmd.api import main
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2auth/cmd/api.py", line 23, in <module>
    from st2common.service_setup import setup as common_setup
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/service_setup.py", line 36, in <module>
    from st2common.util.debugging import enable_debugging
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/debugging.py", line 20, in <module>
    import paramiko
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/paramiko/__init__.py", line 22, in <module>
    from paramiko.transport import SecurityOptions, Transport
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/paramiko/transport.py", line 129, in <module>
    class Transport(threading.Thread, ClosingContextManager):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/paramiko/transport.py", line 190, in Transport
    if KexCurve25519.is_available():
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/paramiko/kex_curve25519.py", line 30, in is_available
    X25519PrivateKey.generate()
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/primitives/asymmetric/x25519.py", line 44, in generate
    return backend.x25519_generate_key()
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 2227, in x25519_generate_key
    evp_pkey = self._evp_pkey_keygen_gc(self._lib.NID_X25519)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 2215, in _evp_pkey_keygen_gc
    self.openssl_assert(evp_pkey_ctx != self._ffi.NULL)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 131, in openssl_assert
    return binding._openssl_assert(self._lib, ok)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 78, in _openssl_assert
    errors_with_text
cryptography.exceptions.InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=101306568, lib=6, func=157, reason=200, reason_text=b'error:0609D0C8:digital envelope routines:int_ctx_new:disabled for FIPS')])

Expected Results

Stackstorm can be deployed successfully on FIPS enabled OCP env

Actual Results

Stackstorm deployment failed FIPS enabled OCP env

Start with the traceback through st2common/util/debugging.py, paramiko, and cryptography's OpenSSL backend, then reproduce the deployment on an FIPS-enabled OCP environment. Compare the failing initialization with the expected behavior; done means StackStorm deploys successfully without the FIPS-related startup failure.