Unit test failures in test_worker and test_workflow_engine on RHEL7 amd64 and RHEL8 ppc64le

SUMMARY

The following two tests are failing:

tests.unit.test_worker.WorkerTestCase.test_worker_shutdown
tests.unit.test_workflow_engine.WorkflowExecutionHandlerTest.test_process

The test_worker failure log exhibits the following error messages repeatedly:

AttributeError: 'NoneType' object has no attribute 'drain_events'
st2.st2common.transport.publishers: WARNING: Error closing channel.

And the test_workflow_engine failure log has the following messages repeating:

amqp.exceptions.InvalidCommand: Channel.open: (503) COMMAND_INVALID - second 'channel.open' seen
st2.st2common.services.workflows: INFO: ......
st2.st2common.transport.publishers: WARNING: Error closing channel.

STACKSTORM VERSION

st2 3.4dev, on Python 3.6.8

commit: 632a07ed7869aa301203e9f9c1193e2693959f08

OS, environment, install method

OS: RHEL 7.9 amd64
OS: UBI/RHEL 8.2 ppc64le
Build from source.

Steps to reproduce the problem

yum install -y python3-pip python3-virtualenv python3-tox gcc-c++ git screen icu libicu libicu-devel openssl-devel make libffi-devel libyaml
yum install -y rh-mongodb36
source scl_source enable rh-mongodb36
systemctl enable rh-mongodb36-mongod
systemctl start rh-mongodb36-mongod
yum install -y rabbitmq-server
systemctl enable rabbitmq-server
systemctl restart rabbitmq-server
ln -sf /usr/bin/virtualenv-3 /usr/bin/virtualenv
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
git clone https://github.com/StackStorm/st2.git
cd st2
git submodule update --init --recursive
make all

Expected Results

All 128 tests should pass.

Actual Results

2/128 tests are failing. Log attached: unit-test.log

Thanks!

Start by running tests.unit.test_worker.WorkerTestCase.test_worker_shutdown and tests.unit.test_workflow_engine.WorkflowExecutionHandlerTest.test_process on the reported RHEL7 amd64 and RHEL8 ppc64le environments, then review unit-test.log and the repeated AMQP errors. Trace the worker shutdown and workflow channel handling until the failures are understood; done means all 128 unit tests pass.