saltstack / saltstack/salt

[BUG] salt-minion 3005.1 on Ubuntu 22.04 hangs when /etc/ssl/openssl.cnf contains QAT Engine configuration

Open
#63,192 1 comment 0 reactions 1 assignee View on GitHub

@Akm0d is already working on this.

Since Mar 21, 2024.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Salt commands (salt-minion, salt-call or /opt/saltstack/salt/run/run) hang with no log output at all, not a single line with -l debug or -l all. Removing salt-minion package and trying to reinstall also fails (since it attempts to start salt-minion service during installation).

Looks like when we delete /etc/ssl/openssl.cnf, or just remove the qatengine configuration from openssl.cnf, it works again (See below for the setup).

We don't have this issue with salt 3004.1, if I just remove salt-minion 3005.1 on the same machine and install 3004.1, it just works normally.

We can reproduce the problem on bare metal servers or Amazon EC2 instances.

Setup

  • Both bare metal servers on-prem and AWS EC2 instances have the problem
  • onedir packaging

We use Intel's Quick Assist Technology accelerators in software mode (qat_sw), which consists of:

  • intel-crypto-mb (2021.6)
  • intel-ipsec-mb (1.3)
  • QAT Engine (0.6.17)
  • OpenSSL 3.0.2
  • Ubuntu 22.04

with the following content in openssl.cnf:

/etc/ssl/openssl.cnf content
openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
qatengine = qatengine_section

[qatengine_section]
engine_id = qatengine
default_algorithms = ALL

Steps to Reproduce the behavior
Install qat_sw components, put the content above into openssl.cnf, restart salt-minion.

Expected behavior
salt-minion being responsive.

Versions Report

salt --versions-report
Salt Version:
          Salt: 3005.1

Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: 1.5.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.11.1
        Python: 3.9.15 (main, Nov  8 2022, 03:42:58)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 22.04 jammy
        locale: utf-8
       machine: x86_64
       release: 5.15.0-1026-aws
        system: Linux
       version: Ubuntu 22.04 jammy

What can I do to further investigate why this happens? Thanks in advance for any help.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.