oxidecomputer / oxidecomputer/omicron

clickhouse query log entries could include more context

Open
#3,012 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Debugging Good for new hires
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

We log a bunch of useful stuff when we make queries to clickhouse (like the SQL that we're executing), but those log messages are lacking context about why we're doing it. It'd be nice if we could preserve the information that's in the logger in the OpContext where we're making the query. That way people can correlate specific external requests (by request id) with specific Clickhouse queries.

This is a little tricky in that we use a client logger today that presumably also has useful key-value pairs (like the clickhouse IP we're connected to?) and I don't know if there's a way to merge two loggers' key-value pairs. But in the worst case we could store the key-value pairs from one of these and by hand extend the other logger with those key-value pairs.

(for an example where this would be useful, see #2994)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the OpContext used at ClickHouse query sites and inspect how the client logger and existing query log fields are assembled. Compare the useful context described here with example #2994; done means correlating an external request ID with a specific ClickHouse query while retaining relevant client-logger context.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, rust
Domain
backend, databases, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.