StackStorm / StackStorm/st2-packages

Add systemd startup order dependencies

Open
#531 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Shell
Stars
28
Forks
63
PR merge metrics
No merged PRs in 30d

Description

Problem

During the https://github.com/StackStorm/ova/pull/68 work, we've found it's a common case when services are started during machine boot time in not the best order, eg. st2api is up and accepting connections, when RabbitMQ didn't start yet, resulting in errors and lost requests.

NB! This will affect only service startup order (which are enabled) during the machine boot stage. Same for shutdown.

Solution

In systemd, there is a rich mechanism to control service startup order via Wants, After, Requires.

https://wiki.archlinux.org/index.php/DeveloperWiki:Systemd says:

The most typical case is that A requires the service B to be running before A is started. In that case add Requires=B and After=B to A.
If the dependency is optional then add Wants=B and After=B instead

https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Wants= says:

A weaker version of Requires=. Units listed in this option will be started if the configuring unit is. However, if the listed units fail to start or cannot be added to the transaction, this has no impact on the validity of the transaction as a whole. This is the recommended way to hook start-up of one unit to the start-up of another unit.

"Relaxed" unit dependency sounds exactly what's needed, because services like RabbitMQ/MongoDB could be deployed on external systems, HA-friendly, while nginx could be not present locally at all.


This should be well tested in both single-machine, as well as HA-emulated deployment to avoid any kind of regressions and corner cases. If regression is unavoidable, - skip this effort.

As a result it will improve st2 service reliability for a single-machine deployments.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the systemd unit definitions for st2api and the related RabbitMQ, MongoDB, and nginx services, then review how Wants, After, and Requires could express their relationships. Test service boot and shutdown ordering in both single-machine and HA-emulated deployments; done means avoiding startup errors and lost requests without breaking external-service deployments.

Written by the indexing model from the issue text.

Assessment

Domain
infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.