quickwit-oss / quickwit-oss/quickwit

Log network latency breakdow somehow.

Open
#1,594 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

Problem

Today we do not really understand how long is spent in the different phase of a data request.

Ideally we would like a breakdown of

did we need to establish a connection
how long did connection establishment (if there was one)
how long does TLS handshake took
time elapsed between sending the request and receiving the first byte. (TTFB)
Once we have this information, we need to try and see if we can lower the latency (connection pool misconfigured, TCP::NO_DELAY).

Notes

Getting that logging will be a bit tricky.
Rusoto let's us plug in any client, http::Client is the one that we us (and it is the default).

http::Client currently does not log all of this information, but maybe we could contribute this upstream,
or wrap http::Client and wrap the result Stream to log the first byte.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining how Rusoto uses the default http::Client and how request streams expose connection establishment, TLS timing, and time to first byte. Compare contributing the missing instrumentation upstream with wrapping http::Client and the result Stream. Done means selecting an approach and recording the requested latency breakdown, then verifying it on representative data requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.