oxidecomputer / oxidecomputer/omicron
test failure: `oximeter-db::integration_test test_cluster`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
On branch ben/poison-clickhouse-connections-on-failure commit a83396a37, I saw this test failure:
bnaecker@flint : ~/file-cabinet/oxide/omicron $ cargo nextest run -p oximeter-db
info: experimental features enabled: setup-scripts, benchmarks
Blocking waiting for file lock on build directory
Compiling oximeter-db v0.1.0 (/Users/bnaecker/file-cabinet/oxide/omicron/oximeter/db)
Compiling nexus-types v0.1.0 (/Users/bnaecker/file-cabinet/oxide/omicron/nexus/types)
Compiling nexus-config v0.1.0 (/Users/bnaecker/file-cabinet/oxide/omicron/nexus-config)
Compiling omicron-test-utils v0.1.0 (/Users/bnaecker/file-cabinet/oxide/omicron/test-utils)
Compiling oximeter-test-utils v0.1.0 (/Users/bnaecker/file-cabinet/oxide/omicron/oximeter/test-utils)
Finished `test` profile [unoptimized + debuginfo] target(s) in 10.35s
────────────
Nextest run ID f80c4026-074d-483e-88f6-ba43855eaeb5 with nextest profile: default
Starting 255 tests across 3 binaries
PASS [ 0.012s] oximeter-db client::oxql::tests::test_chunk_consistent_key_groups_split_middle_of_key_group
PASS [ 0.013s] oximeter-db client::oxql::tests::test_chunk_consistent_key_groups_split_key_group_multiple_times
...
PASS [ 1.355s] oximeter-db oxql::ast::grammar::tests::test_recognize_escape_sequence_with_valid_unicode_sequence
FAIL [ 7.456s] oximeter-db::integration_test test_cluster
stdout ───
running 1 test
Deploying keeper: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.1-oximeter-clickward-test/keeper-1
Deploying keeper: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.1-oximeter-clickward-test/keeper-3
Deploying keeper: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.1-oximeter-clickward-test/keeper-2
Deploying clickhouse server: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.1-oximeter-clickward-test/clickhouse-1
Deploying clickhouse server: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.1-oximeter-clickward-test/clickhouse-2
test test_cluster ... FAILED
failures:
failures:
test_cluster
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 7.44s
stderr ───
log file: /var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.0.log
note: configured to log to "/var/folders/67/2tlym22x1r3d2kwbh84j298w0000gn/T/integration_test-d5a1d0774d2cfe91-test_cluster.32777.0.log"
thread 'test_cluster' (74469256) panicked at oximeter/db/tests/integration_test.rs:277:5:
expected non-zero UserTimeMicroseconds in profile summaries, got 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
This assertion was introduced in #10172. In that PR, @jmcarp noted that the profile event messages with the relevant value, UserTimeMicroseconds, is occassionally missing, in which case we default to returning 0 for it. That trips this assertion, by construction. We should probably either remove this assertion or return a map of Option<i64>s from profile_summary() to distinguish 0 from "missing".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with oximeter/db/tests/integration_test.rs at the test_cluster assertion around line 277, then inspect profile_summary() and the profile event handling it uses. Reproduce the test with cargo nextest run -p oximeter-db, and make the behavior distinguish a missing UserTimeMicroseconds value from a real zero so test_cluster no longer fails for missing data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, rust
- Domain
- databases, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100