oxidecomputer / oxidecomputer/omicron

oxlog does not find archived live log files

Open
#9,271 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

#9226 added archival of live log files from zones when they shut down and when the sled reboots. But they get named slightly differently and oxlog doesn't find them. After a recent dogfood update:

BRM27230045 # ls -1 /pool/ext/*/crypt/debug/*nexus_dcb*/*nexus* > all_logs.txt

BRM27230045 # /opt/oxide/oxlog/oxlog logs oxz_nexus_dcb27edf-5a97-459a-92c0-63d795744055 nexus --archived > oxlog_logs.txt

BRM27230045 # diff oxlog_logs.txt all_logs.txt 
137a138,139
> /pool/ext/8a0016f3-3cdc-4b4c-b1d3-5334dd365465/crypt/debug/oxz_nexus_dcb27edf-5a97-459a-92c0-63d795744055/oxide-nexus:default.1761086667
> /pool/ext/8a0016f3-3cdc-4b4c-b1d3-5334dd365465/crypt/debug/oxz_nexus_dcb27edf-5a97-459a-92c0-63d795744055/oxide-nexus:default.1761098396

There are two choices here:

  • teach oxlog about these paths. (pro: will cover files already archived with this problem. risk: is there some other file in these datasets that it will mistakenly pick up?)
  • change sled agent to preserve the .log in this case. This seems better, but would require more testing (this code is not factored in a way that makes it testable without fully running it all, including archiving on a deployed system) and oxlog would never pick up log files that had been archived without this fix.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the oxlog logs command and the archived path patterns shown in the issue, then compare them with the sled agent archival behavior introduced by #9226. Decide whether oxlog should recognize the existing filenames or sled agent should preserve the .log suffix; done means archived live logs are found without unrelated files being included, with testing covering the chosen behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.