so-Influxdb missing container on update

__Steps to reproduce:__
List the minimal actions needed to reproduce the behaviour.

architecture: Linux 5.15.0-204.147.6.2.el9uek.x86_64 x86_64

During the update from 2.4.50 to 2.4.60 we are getting hung on the following command:

Executing command '/usr/sbin/so-influxdb-manage' in directory '/root'

Currently troubleshooting, making a note for a future solve to this problem.

1st Attempt at solve:

accessed ssl certifcate and key within the manger node /etc/pki/managerssl.xxx
Copied the key and cert in to SOS config under > nginx > ssl > SSL/TSL xxx
Retrying the update...

ts=2024-04-08T00:05:12.208738Z lvl=info msg="http: TLS handshake error from 127.0.0.1:34684: EOF" log_id=0o5f0DvW000 service=http

After trying to run soup again, we have identified another issue where the salt minions are no longer talking to the salt-master (Manager Node).

I am deleting the keys and reaccepting them. To do this:
> On the manager execute: salt-key
> Run salt-key -d this deletes all the current keys
> Restart the minions (Search, Receiver, Forward)
> Go back to the manager and run salt-key again
> Accept the keys by executing: salt-key -A
> Run salt\* test.ping to confirm everything is working again

The salt is now working on all the nodes but we still have the following error when trying to run soup
- Specified SLS 'data.*' in environment 'base' is not available on the salt master
- Specified SLS 'global' in environment 'base' is not available on the salt master

Found a GitHub with a similar problem
> Trying to run salt * saltutil.refresh_pillar
> Have to replace * with the name of each minion
> And running salt * saltutil.sync_grains
> Again have to replace *

This did not resolve the issue. Same error is occurring.

As I understand now, the file global.sls does not exist where it should. I started a GitHub discussion to try and resolve the missing file(s) and am waiting for a response.

---------------------------------------------------------

Update: I was able to get through the soup command. I now know the issue was not with the script but rather an issue with the influxdb server. When we try to run salt \* state.highstate we get hungup on the same command. If you wait long enough it will error out and keep running. The error codes I could find in the influxdb.log are below: *Also included is a screenshot of the error log.

ID: influxdb-setup
Function: cmd.run
Name: /usr/sbin/so-influxdb-manage setup &>> /opt/so/log/influxdb/setup.log
Result: False
Comment: Command "/usr/sbin/so-influxdb-manage setup &>> /opt/so/log/influxdb/setup.log" run
Started: 19:38:12.399766
Duration: 302493.461 ms
Changes:
----------
pid: 83801
retcode: 1
stderr:
stdout:
ID: metrics_link_file
Function: cmd.run
Name: so-influxdb-manage dashboardpath "Security Onion Performance" > /opt/so/saltstack/local/salt/influxdb/metrics_link.txt
Result: False
Comment: Command "so-influxdb-manage dashboardpath "Security Onion Performance" > /opt/so/saltstack/local/salt/influxdb/metrics_link.txt" run
Started: 19:43:15.145821
Duration: 301277.019 ms
Changes:
----------
pid: 103999
retcode: 1
stderr:

Thu Apr 11 19:43:15 UTC 2024 | so-influxdb-manage | Server does not appear to be running or fully initialized - will try again in 10 seconds (1 / 30)

---------------------------

We are rolling back to the previous version and accepting the outdated version until there is a fix for the issue above. If we keep the update as is, influxdb will not work.

__Expected behaviour:__
We expected to run soup and just update our nodes to the latest version.

__Actual behaviour:__
so-status (see all containers) run soup, so-status, then so-influxdb is missing as a container.
We ran soup, so-rule-update, salt \*state.highstate etc

__Environment info:__

Linux 5.15.0-204.147.6.2.el9uek.x86_64 x86_64

__Config:__
Copy any non-default config values here or attach the full config as a gist or file.

__Logs:__
Include snippet of errors in logs or stack traces here.
Sometimes you can get useful information by running the program with the `RUST_BACKTRACE=full` environment variable.
Finally, the IOx server has a `-vv` for verbose logging.

Start with the /usr/sbin/so-influxdb-manage entry point and the salt state.highstate steps described in the report. Review the setup.log and influxdb.log output around influxdb-setup and metrics_link_file, then reproduce the 2.4.50-to-2.4.60 update. Done means the update completes and so-influxdb remains available as a container.

Written by the indexing model from the issue text.