BUG: metrics "opendal_operation_duration_seconds_count" gets HTTP request count for chunk reads
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Hi team, I notice for simple 1-1 mapping read + write operations (i.e., read an object from bucket A and write it to bucket B), read and write request rate shows completely different results (read RPS is several times larger than write RPS), both are backed by metrics `opendal_operation_duration_seconds_count`.
If I don't read it wrong:
- For read operations, the metrics is actually counting for HTTP requests for chunk reader
+ On read operations, we create multiple readers, with each of them containing the full layers: https://github.com/apache/opendal/blob/69114d2715ce3863b77b84392050aa1c3dc85037/core/core/src/types/read/buffer_stream.rs#L115
- For write operations, things are different, all writes through the same writer created here https://github.com/apache/opendal/blob/69114d2715ce3863b77b84392050aa1c3dc85037/core/core/src/types/context/write.rs#L113
- In one word, for read operations, the operation count metrics is measuring the HTTP request count, while for write operations, it's measuring the write operation/function call count.
It's not ideal from my perspective:
- Read and write operation counts are completely different if I call `Operation::read` and `Operation::write` the same time, which causes user confusion
- Read operation count overlaps with HTTP request count
### Problem and Solution
I made my workaround to collect and emit these metrics at application level.
But would be good to calculate these two metrics in the same way for both operations.
### Additional Context
_No response_
### Are you willing to contribute to the development of this feature?
- [ ] Yes, I am willing to contribute to the development of this feature.
Contributor guide
Research direction
Start by tracing the read path in core/core/src/types/read/buffer_stream.rs around line 115 and the write path in core/core/src/types/context/write.rs around line 113, then locate where opendal_operation_duration_seconds_count is recorded. Compare how read and write operations contribute to the metric and verify that the resulting counts use consistent operation-level semantics without counting chunk HTTP requests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100