0xMiden / 0xMiden/note-transport-service

RequestTimer double-counts requests; no error-rate or saturation metrics

Open
#128 1 comment 0 reactions 0 assignees View on GitHub
bug production-readiness
Dominant language
Rust
Stars
3
Forks
10
Avg merge
2h 23m
Merged PRs (30d)
4

Description

Severity: high (bug + missing coverage).

### Summary

Two related observability defects.

**a) Duration histograms are double-counted and the only error signal is meaningless.** `RequestTimer::finish(&self, …)` takes `&self` and sets no completed flag (`crates/node/src/metrics.rs:278`), and `Drop` unconditionally calls `self.finish("dropped")` (`crates/node/src/metrics.rs:293-297`). Every success path calls `finish("ok")` and *then* the drop records the same duration again with `status="dropped"`. Result: sample counts are 2x inflated and `status="dropped"` fires on every successful request, so error rate and latency percentiles are wrong.

**b) No metrics an operator would alert on.** `crates/node/src/metrics.rs` defines only per-RPC counters/histograms. Missing: gRPC error counter by status code, active `stream_notes` subscription count, total note count, DB file size, connection-pool utilization, maintenance last-success heartbeat, process build-info/uptime.

### Recommendation

- Make `finish(self, …)` consume the timer (or guard `Drop` with a `finished: bool`) and record error paths with a real `status="error"`.
- Add an error counter with a `code` label (ideally a tower/tonic interceptor so it covers all RPCs), plus observable gauges for subscriptions, note count, DB size, pool state, and a maintenance heartbeat.

---
Part of #114.

Contributor guide

Open the contributing guide

Research direction

The bug is in crates/node/src/metrics.rs around lines 278 and 293-297. Start by examining the RequestTimer struct and its finish and Drop implementations. To verify the fix, run any existing tests for metrics. Adding new metrics requires understanding the gRPC setup and the codebase's structure for counters and gauges.

Written by the indexing model from the issue text.

Assessment

Domain
backend, observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.