apache / apache/opendal

BUG: metrics "opendal_operation_duration_seconds_count" gets HTTP request count for chunk reads

Open
#7,339 1 comment 1 reaction 0 assignees View on GitHub
core enhancement releases-note/feat
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

### Feature Description

Hi team, I notice for simple 1-1 mapping read + write operations (i.e., read an object from bucket A and write it to bucket B), read and write request rate shows completely different results (read RPS is several times larger than write RPS), both are backed by metrics `opendal_operation_duration_seconds_count`.

If I don't read it wrong:
- For read operations, the metrics is actually counting for HTTP requests for chunk reader
+ On read operations, we create multiple readers, with each of them containing the full layers: https://github.com/apache/opendal/blob/69114d2715ce3863b77b84392050aa1c3dc85037/core/core/src/types/read/buffer_stream.rs#L115
- For write operations, things are different, all writes through the same writer created here https://github.com/apache/opendal/blob/69114d2715ce3863b77b84392050aa1c3dc85037/core/core/src/types/context/write.rs#L113
- In one word, for read operations, the operation count metrics is measuring the HTTP request count, while for write operations, it's measuring the write operation/function call count.

It's not ideal from my perspective:
- Read and write operation counts are completely different if I call `Operation::read` and `Operation::write` the same time, which causes user confusion
- Read operation count overlaps with HTTP request count

### Problem and Solution

I made my workaround to collect and emit these metrics at application level.
But would be good to calculate these two metrics in the same way for both operations.

### Additional Context

_No response_

### Are you willing to contribute to the development of this feature?

- [ ] Yes, I am willing to contribute to the development of this feature.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the read path in core/core/src/types/read/buffer_stream.rs around line 115 and the write path in core/core/src/types/context/write.rs around line 113, then locate where opendal_operation_duration_seconds_count is recorded. Compare how read and write operations contribute to the metric and verify that the resulting counts use consistent operation-level semantics without counting chunk HTTP requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.