oxidecomputer / oxidecomputer/omicron

test-flake: omicron-nexus::test_all integration_tests::oximeter::test_oximeter_reregistration

Open
#4,945 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Test Flake
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

The test

 omicron-nexus::test_all integration_tests::oximeter::test_oximeter_reregistration

Is failing for me intermittently running the test suite locally.

--- STDERR:              omicron-nexus::test_all integration_tests::oximeter::test_oximeter_reregistration ---
log file: /tmp/test_all-1d231510be55a0d6-test_oximeter_reregistration.25361.0.log
note: configured to log to "/tmp/test_all-1d231510be55a0d6-test_oximeter_reregistration.25361.0.log"
DB URL: postgresql://root@[::1]:44012/omicron?sslmode=disable
DB address: [::1]:44012
log file: /tmp/test_all-1d231510be55a0d6-test_oximeter_reregistration.25361.2.log
note: configured to log to "/tmp/test_all-1d231510be55a0d6-test_oximeter_reregistration.25361.2.log"
thread 'integration_tests::oximeter::test_oximeter_reregistration' panicked at nexus/tests/integration_tests/oximeter.rs:254:6:
There should be new data after restarting the producer: TimedOut(30.04522333s)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
WARN: temporary directory leaked: /tmp/.tmpQiCvIf
WARN: dropped ClickHouseInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
failed to clean up ClickHouse data dir:
- /tmp/test_all-1d231510be55a0d6-test_oximeter_reregistration.25361.1-clickhouse-G72Q2E: File exists (os error 17)
WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
WARN: dendrite temporary directory leaked: /tmp/.tmp9uhIby
WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
WARN: dendrite temporary directory leaked: /tmp/.tmpMysld8
WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
WARN: mgd temporary directory leaked: /tmp/.tmpwwYSzK
WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
WARN: mgd temporary directory leaked: /tmp/.tmptCfMoR

   Canceling due to test failure: 15 tests still running

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at nexus/tests/integration_tests/oximeter.rs:254 and run integration_tests::oximeter::test_oximeter_reregistration, using the linked test and ClickHouse logs to investigate the intermittent timeout. Done means the test reliably observes new data after restarting the producer without timing out.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, postgresql, rust
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.