googleapis / googleapis/google-cloud-rust

Capture stream lifetime errors in tracing spans

Open
#5,372 0 comments 0 reactions 0 assignees View on GitHub
type: feature request
Dominant language
Rust
Stars
955
Forks
144
Avg merge
1d 5h
Merged PRs (30d)
279

Description

The current implementation of observability for streaming RPCs (specifically in `bidi_stream_with_status` and `server_streaming_with_status` in `gax-internal/src/grpc.rs`) only instruments the future that establishes the stream using `WithTransportSpan`.

This means:
1. The span only measures the duration of the *initiation* of the stream.
2. The span is closed with `OK` status as soon as the stream is successfully opened.
3. Any errors that occur later during the streaming phase (e.g., when reading/writing messages) are not captured on the span.

This creates a problem: any errors that occur after the stream is successfully opened (during data transmission) are not recorded on the span. The span remains marked as successful even if the stream later fails.

This was observed in the `grpc_observability.rs` tests (`streaming_error`), where a stream error caused by an invalid message did not result in the span being marked as an error because the connection itself already succeeded.

We should instrument the stream itself (not just the initial connection future) so that errors encountered during the stream's lifetime are recorded on the span. This might involve wrapping the returned stream in a custom type that can intercept errors and update the span status accordingly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.