oxidecomputer / oxidecomputer/dendrite

dendrite shouldn't crash on svcadm refresh

Open
#59 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
20
Forks
3
Avg merge
3d 11h
Merged PRs (30d)
4

Description

On dogfood, sled-agent updated dpd's SMF properties and issued a reset. This sends dpd a SIGUSR1. When the daemon is fully up, we catch the signal and reload the SMF properties from the database. As the daemon is coming up, the signal should be ignored. Here we can see that it actually caused the daemon to crash:

DMA pool BF_DMA_TM_WRITE_LIST_1_Pool: va range: 0xfffff5ffa8800000 -> 0xfffff5ffa9200000  size=0xa00000  page_size: 0x200000
^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\^M|^M/^M-^M\[ Dec 28 00:03:05 Rereading configuration. ]
[ Dec 28 00:03:05 Executing refresh method (:kill). ]
[ Dec 28 00:03:05 Stopping because all processes in service exited. ]
[ Dec 28 00:03:05 Executing stop method (:kill). ]
[ Dec 28 00:03:05 Executing start method ("/opt/oxide/dendrite/bin/svc-dpd"). ]
Using SDE runtime support at: /opt/oxide/bf_sde
[ Dec 28 00:03:05 Method "start" exited with status 0. ]
00:03:05.852Z INFO dpd: dpd config: Config {
        log_file: None,

Without digging deeper, I'm not sure why we're ignoring the signal here, or if that might cause us to run with stale data. If we don't do the initial SMF read until after this point we should be fine. If we've already loaded the SMF config, that seems like a problem.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing dpd's SIGUSR1 handling, startup sequence, and initial SMF configuration read, then reproduce the svcadm refresh timing shown in the issue. Confirm that a signal during startup cannot crash the daemon, while a signal after startup still reloads SMF properties without leaving stale data.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.