litespeedtech / litespeedtech/openlitespeed

OpenLiteSpeed reloads without lswsctrl

Open
#477 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.5k
Forks
233
Avg merge
2d 19h
Merged PRs (30d)
5

Description

# Problem description

We run automated tests with various configurations to make sure that all the settings and configs work as expected (related to our software). We've noticed that OLS give flacky results on **all rhel8 systems - centos8, alma8, rhel8 and rocky8,** - the configs do not seem to be reloaded reliably.

Digging deeper revealed that the main issue is with the **systemd** itself, where a reload command in parallel not only gets ignored, but also breaks the initial reload command.

## Working state

```
[root@localhost system]# date; systemctl reload openlitespeed; date
Wed Apr 22 20:15:52 EEST 2026
Wed Apr 22 20:15:55 EEST 2026
[root@localhost system]# date; systemctl reload openlitespeed; date
Wed Apr 22 20:15:58 EEST 2026
Wed Apr 22 20:16:01 EEST 2026
```

Reloads are confirmed in journal:
```
Apr 22 20:15:52 localhost.localdomain systemd[1]: Reloading OpenLiteSpeed HTTP Server.
Apr 22 20:15:52 localhost.localdomain lswsctrl[19937]: [OK] Send SIGUSR1 to 19924
Apr 22 20:15:55 localhost.localdomain systemd[1]: Reloaded OpenLiteSpeed HTTP Server.
Apr 22 20:15:58 localhost.localdomain systemd[1]: Reloading OpenLiteSpeed HTTP Server.
Apr 22 20:15:58 localhost.localdomain lswsctrl[19959]: [OK] Send SIGUSR1 to 19947
Apr 22 20:16:01 localhost.localdomain systemd[1]: Reloaded OpenLiteSpeed HTTP Server.
```

## Broken reloads

When a parallel reload gets issued:
```
[root@localhost system]# date; systemctl reload openlitespeed; date
Wed Apr 22 20:18:05 EEST 2026
Wed Apr 22 20:18:07 EEST 2026
+in parralel:
[root@localhost ~]# date; systemctl reload openlitespeed; date
Wed Apr 22 20:18:07 EEST 2026
Wed Apr 22 20:18:07 EEST 2026
```

Journal has only the beginning of the reload, but not the end:
```
Apr 22 20:18:05 localhost.localdomain systemd[1]: Reloading OpenLiteSpeed HTTP Server.
Apr 22 20:18:05 localhost.localdomain lswsctrl[20064]: [OK] Send SIGUSR1 to 20048
```

Basically, the first command vanishes when another command gets executed in parallel.
Timestamps show exactly the same exit time for both reload commands.

## OLS Native systemd?

We haven't noticed this buggy systemd behavior with other services due to the very simple fact - **reloads usually are just one signal sent to a service** to reload the configs, which basically is instant, but OLS uses lswsctrl script for reload that has sleep commands hardcoded:

The OpenLiteSpeed systemd script `/etc/systemd/system/openlitespeed.service` uses lswsctrl script to reload the service:
```
ExecReload=/usr/local/lsws/bin/lswsctrl restart
```

So the best case for reload when everything is working:
2 seconds here:
https://github.com/litespeedtech/openlitespeed/blob/c6076315a598a79e3f8edc663390b458e27b1710/dist/bin/lswsctrl#L191
and 1 second here:
https://github.com/litespeedtech/openlitespeed/blob/c6076315a598a79e3f8edc663390b458e27b1710/dist/bin/lswsctrl#L165
Which is always 3 seconds minimum.

## Suggestion/Request

The main question is - can lswsctrl script be eliminated by using systemd natively?

Our tested example:
```
# should be added as /etc/systemd/system/openlitespeed.service
[Unit]
Description=The OpenLiteSpeed HTTP Server
After=network-online.target remote-fs.target nss-lookup.target lve_namespaces.service
Wants=network-online.target

[Service]
Type=forking
WorkingDirectory=/usr/local/lsws
PIDFile=/run/openlitespeed.pid
ExecStart=/usr/local/lsws/bin/openlitespeed
Restart=always
ExecReload=/bin/kill -USR1 ${MAINPID}
SuccessExitStatus=255

PrivateTmp=false
LimitMEMLOCK=infinity
LimitNOFILE=655350

[Install]
WantedBy=multi-user.target
Alias=lsws.service
Alias=lshttpd.service
Alias=litespeed.service
```

## Nuances with systemd

The start/restart/reload seems to be working as expected with two exceptions that are worying:

1. We must use custom `SuccessExitStatus` due to some guardCrash and poll related return code, where the exit code 0 on successful exit would be preferred
```
SuccessExitStatus=255
```
otherwise we get:
```
Apr 23 07:52:49 localhost.localdomain systemd[1]: lshttpd.service: Main process exited, code=exited, status=255/n/a
Apr 23 07:52:49 localhost.localdomain systemd[1]: lshttpd.service: Failed with result 'exit-code'.
````

2. When starting litespeed, pid file gets created in a later stage than expected:
```
Apr 22 11:49:51 localhost.localdomain systemd[1]: Starting The OpenLiteSpeed HTTP Server...
Apr 22 11:49:51 localhost.localdomain systemd[1]: lshttpd.service: Can't open PID file /run/openlitespeed.pid (yet?) after start: No such file or directory
Apr 22 11:49:51 localhost.localdomain systemd[1]: Started The OpenLiteSpeed HTTP Server.
```

We would be very grateful to see native OLS systemd implementation.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the openlitespeed.service unit and the referenced dist/bin/lswsctrl sections at lines 165 and 191. Reproduce parallel systemctl reload calls on a RHEL 8 system, then compare the native systemd behavior with the script-based behavior, including journal output, SuccessExitStatus, and PID-file startup timing. Done means reloads are reliable and service start status is correctly reported.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, shell
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.