[BUG] Minions keep disconnecting from Master with timeout errors.

Description
Minions keep losing connection to the sale master. There doesn't seem to be a pattern as to which minions are impacted. At no point are all minions disconnected. Example: I'll run "test.ping" on a nodegroup that has 10 minions in it, and 4 will return successfully, 6 will fail. Another nodegroup will have different results. Restarting the salt-minion service corrects the issue and restores connectivity. At least until some point in the future, may be 24 hours, may be over a week.

I have verified that the issue is not backed on the salt-master having high-load. Master is 8vCPU & 6vGB RAM, and has an overall load average of less than 1, and plenty of ram left in buffers/cache and available.

Setup
Simple salt environment. 1 master, 144 minions.
Minions: OS is "Sangoma Linux release 7.8.2003 (Core)". ( In our other environments the minions are running RedHat 8.)
Master: OS is "Red Hat Enterprise Linux release 8.9 (Ootpa)"

Config files....
I have attached the modified files. many (such as /etc/salt/minion, /etc/salt/proxy) are unchanged from what was deployed by the installer. Our changes go into /etc/salt/minion.d/.
(Config files renamed from .conf to .txt so this page would allow attachment.)
00-hash.txt
00-master.txt
10-pkg-manager.conf.txt
20-logging.txt

(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Please be as specific as possible and give set-up details.

• [X ] VM running on a cloud service, please be explicit and add details
  Salt-master is a VM instance running on GCP. We have this exact same config utilized in two other separate GCP projects (as well as locally in my vagrant lab) without issue. One of those environments is larger, and has much more interactions with the minions than this one that is having the issues.

Minions: OS is "Sangoma Linux release 7.8.2003 (Core)". ( In our other environments the minions are running RedHat 8.)
Master: OS is "Red Hat Enterprise Linux release 8.9 (Ootpa)"

Steps to Reproduce the behavior
Command...
salt -N nodegroup_1 test.ping

Response...
mv-minion1.testdoman.internal:
True
mv-minion2.testdoman.internal:
True
mv-minion3.testdoman.internal:
'test' virtual returned False: cannot import name 'DEFAULT_HASH_TYPE' from 'salt.config' (/opt/saltstack/salt/lib/python3.10/site-packages/salt/config/init.py)
mv-minion4.testdoman.internal:
'test' virtual returned False: cannot import name 'DEFAULT_HASH_TYPE' from 'salt.config' (/opt/saltstack/salt/lib/python3.10/site-packages/salt/config/init.py)
mv-minion5.testdoman.internal:
'test' virtual returned False: cannot import name 'DEFAULT_HASH_TYPE' from 'salt.config' (/opt/saltstack/salt/lib/python3.10/site-packages/salt/config/init.py)
mv-minion6.testdoman.internal:
'test' virtual returned False: cannot import name 'DEFAULT_HASH_TYPE' from 'salt.config' (/opt/saltstack/salt/lib/python3.10/site-packages/salt/config/init.py)

(Include debug logs if possible and relevant)
Here is an example event chain when attempting to run the "test.ping" command against this minion. (Output has been anonomyzed.)

Jan 3 11:16:17 mv-minion4 salt-minion: [INFO ] User sudo_johnsmith_mv_test Executing command test.ping with jid 20240103171617897385
Jan 3 11:16:18 mv-minion4 salt-minion: [INFO ] Starting a new job 20240103171617897385 with PID 16798
Jan 3 11:16:19 mv-minion4 salt-minion: [INFO ] Returning information for job: 20240103171617897385
Jan 3 11:16:19 mv-minion4 salt-minion: [INFO ] Payload: {"index": "main", "sourcetype": "saltjson", "event": {"success": false, "return": "'test' virtual returned False: cannot import name 'DEFAULT_HASH_TYPE' from 'salt.config' (/opt/saltstack/salt/lib/python3.10/site-packages/salt/config/init.py)", "retcode": 1, "out": "nested", "jid": "20240103171617897385", "fun": "test.ping", "fun_args": [], "user": "sudo_johnsmith_mv_test", "id": "mv-minion4.testdomain.internal"}}
Jan 3 11:17:38 mv-minion4 salt-minion: [ERROR ] The return failed for job 20240103171528683117: HTTPSConnectionPool(host='salt-master.testdomain.internal', port=8088): Max retries exceeded with url: /services/collector/event (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f7a61241ff0>, 'Connection to salt-master.testdomain.internal timed out. (connect timeout=None)'))
Jan 3 11:17:38 mv-minion4 salt-minion: Traceback (most recent call last):
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connection.py", line 203, in _new_conn
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
Jan 3 11:17:38 mv-minion4 salt-minion: sock.connect(sa)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
Jan 3 11:17:38 mv-minion4 salt-minion: af, socktype, proto, canonname, sa = res
Jan 3 11:17:38 mv-minion4 salt-minion: TimeoutError: [Errno 110] Connection timed out
Jan 3 11:17:38 mv-minion4 salt-minion: The above exception was the direct cause of the following exception:
Jan 3 11:17:38 mv-minion4 salt-minion: Traceback (most recent call last):
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
Jan 3 11:17:38 mv-minion4 salt-minion: SSLError(e),
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
Jan 3 11:17:38 mv-minion4 salt-minion: self._absolute_url(url),
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
Jan 3 11:17:38 mv-minion4 salt-minion: six.raise_from(e, None)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connection.py", line 611, in connect
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connection.py", line 212, in _new_conn
Jan 3 11:17:38 mv-minion4 salt-minion: match = _CONTAINS_CONTROL_CHAR_RE.search(method)
Jan 3 11:17:38 mv-minion4 salt-minion: urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f7a61241ff0>, 'Connection to salt-master.testdomain.internal timed out. (connect timeout=None)')
Jan 3 11:17:38 mv-minion4 salt-minion: The above exception was the direct cause of the following exception:
Jan 3 11:17:38 mv-minion4 salt-minion: Traceback (most recent call last):
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
Jan 3 11:17:38 mv-minion4 salt-minion: resp = conn.urlopen(
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
Jan 3 11:17:38 mv-minion4 salt-minion: redirect_location = redirect and response.get_redirect_location()
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
Jan 3 11:17:38 mv-minion4 salt-minion: :param response: A response object, or None, if the server did not
Jan 3 11:17:38 mv-minion4 salt-minion: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='salt-master.testdomain.internal', port=8088): Max retries exceeded with url: /services/collector/event (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f7a61241ff0>, 'Connection to salt-master.testdomain.internal timed out. (connect timeout=None)'))
Jan 3 11:17:38 mv-minion4 salt-minion: During handling of the above exception, another exception occurred:
Jan 3 11:17:38 mv-minion4 salt-minion: Traceback (most recent call last):
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/minion.py", line 2096, in _thread_return
Jan 3 11:17:38 mv-minion4 salt-minion: ret["success"] = False
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 159, in call
Jan 3 11:17:38 mv-minion4 salt-minion: ret = self.loader.run(run_func, *args, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1245, in run
Jan 3 11:17:38 mv-minion4 salt-minion: return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1260, in _run_as
Jan 3 11:17:38 mv-minion4 salt-minion: return _func_or_method(*args, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/returners/splunk.py", line 62, in returner
Jan 3 11:17:38 mv-minion4 salt-minion: http_collector.sendEvent(payload)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/returners/splunk.py", line 208, in sendEvent
Jan 3 11:17:38 mv-minion4 salt-minion: r = requests.post(
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/api.py", line 115, in post
Jan 3 11:17:38 mv-minion4 salt-minion: return request("post", url, data=data, json=json, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/api.py", line 59, in request
Jan 3 11:17:38 mv-minion4 salt-minion: return session.request(method=method, url=url, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
Jan 3 11:17:38 mv-minion4 salt-minion: resp = self.send(prep, **send_kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
Jan 3 11:17:38 mv-minion4 salt-minion: r = adapter.send(request, **kwargs)
Jan 3 11:17:38 mv-minion4 salt-minion: File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/adapters.py", line 507, in send
Jan 3 11:17:38 mv-minion4 salt-minion: raise ConnectTimeout(e, request=request)
Jan 3 11:17:38 mv-minion4 salt-minion: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='salt-master.testdomain.internal', port=8088): Max retries exceeded with url: /services/collector/event (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f7a61241ff0>, 'Connection to salt-master.testdomain.internal timed out. (connect timeout=None)'))

Expected behavior
Expected "True" responses form all minions in the nodegroup.
Screenshots
NA

Versions Report

PASTE HERE

Additional context
Add any other context about the problem here.

Start with the reported salt/minion.py and salt/returners/splunk.py stack traces, then reproduce with salt -N nodegroup_1 test.ping while reviewing the attached configuration files and minion logs. Done means the affected minions consistently return True without the import or connection-timeout errors.