oxidecomputer / oxidecomputer/omicron

test failed due to non-unique log file

Open
#1,936 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

development Test Flake
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

While running part of the Omicron test suite in a loop, I ran into this:

failures:

---- db::queries::external_ip::tests::test_next_external_ip_is_restricted_to_pools stdout ----
log file: "/dangerzone/omicron_tmp/omicron_nexus-598f187aac60c458-test_next_external_ip_is_restricted_to_pools.3174.55.log"
thread 'db::queries::external_ip::tests::test_next_external_ip_is_restricted_to_pools' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: AlreadyExists, message: "File exists" }', /home/dap/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ed0b9c8/dropshot/src/test_util.rs:476:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    db::queries::external_ip::tests::test_next_external_ip_is_restricted_to_pools

test result: FAILED. 84 passed; 1 failed; 0 ignored; 0 measured; 50 filtered out; finished in 25.10s

error: test failed, to rerun pass `-p omicron-nexus --lib`

The log file has the test name, some Cargo identifier, pid, and a per-process counter. But it still looks like we conflicted with an old log file. I say that because this failure happened in the last 24 hours (so, after 2022-11-10T23:00Z), but that path exists from before that:

 $ ls -lrt /dangerzone/omicron_tmp/omicron_nexus-598f187aac60c458-test_next_external_ip_is_restricted_to_pools.3174.55.log
-rw-r--r--   1 dap      staff        965 Nov  3 19:47 /dangerzone/omicron_tmp/omicron_nexus-598f187aac60c458-test_next_external_ip_is_restricted_to_pools.3174.55.log

This seems probably unlikely in CI, but possible in dev environments where failed test run files may stick around for a while.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at dropshot/src/test_util.rs:476 and reproduce db::queries::external_ip::tests::test_next_external_ip_is_restricted_to_pools, which reports the existing log-file failure. Determine how a stale log file can collide with a new test run, then verify that rerunning tests no longer fails with AlreadyExists.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.