quickwit-oss / quickwit-oss/quickwit
Log network latency breakdow somehow.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Problem
Today we do not really understand how long is spent in the different phase of a data request.
Ideally we would like a breakdown of
did we need to establish a connection
how long did connection establishment (if there was one)
how long does TLS handshake took
time elapsed between sending the request and receiving the first byte. (TTFB)
Once we have this information, we need to try and see if we can lower the latency (connection pool misconfigured, TCP::NO_DELAY).
Notes
Getting that logging will be a bit tricky.
Rusoto let's us plug in any client, http::Client is the one that we us (and it is the default).
http::Client currently does not log all of this information, but maybe we could contribute this upstream,
or wrap http::Client and wrap the result Stream to log the first byte.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining how Rusoto uses the default http::Client and how request streams expose connection establishment, TLS timing, and time to first byte. Compare contributing the missing instrumentation upstream with wrapping http::Client and the result Stream. Done means selecting an approach and recording the requested latency breakdown, then verifying it on representative data requests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100