canonical / canonical/core-base

Support watchdog coverage for first phase of reboot

Open
#71 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
37
Forks
53
Avg merge
1d 19h
Merged PRs (30d)
12

Description

We currently support two system configuration options (since snapd 2.34) that control the behavior of systems with hardware watchdog timers:

* `watchdog.runtime-timeout`
* `watchdog.shutdown-timeout`

The [documentation](https://snapcraft.io/docs/system-options#heading--watchdog-shutdown-timeout) for `shutdown-timeout` says:

> The watchdog shutdown timeout is an interval to permit a clean reboot of the system. If the system fails to reboot within this interval, the watchdog will forcibly restart the system to protect against failed or hanging reboots.

This is slightly misleading, as this timeout is used to reset the hardware watchdog for each iteration of the main loop w/in systemd-shutdown, so in reality the reboot could take much longer than this timeout.

> Note that the shutdown-timeout applies only to the second phase of a reboot, after all regular services are terminated and the system and service manager process has been replaced by the systemd-shutdown binary.

This means that reboot hangs that occur due to misbehaving and/or un-killable processes are not handled by this timeout. The [manpage](https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html) for systemd.conf is a bit confusing as it says:

> During the first phase of the shutdown operation the system and service manager remains running and hence RuntimeWatchdogSec= is still honoured.

...but then it says:

> In order to define a timeout on this first phase of system shutdown, configure JobTimeoutSec= and JobTimeoutAction= in the [Unit] section of the shutdown.target unit.

So it's not 100% clear to me whether we need to additionally modify the `shutdown.target` unit.

Related to this issue is the matter of whether we actually have any test cases to validate watchdog behavior during shutdown.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.