pingcap / pingcap/tiflash

Log format unification for easier grep

Open
#5,726 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Enhancement

As there is no fields support in the current logger, when meeting fields like segment_id, or table_name, we are now outputting them in different ways in the log. For example, it may be like segment [1], or may be like [id:1]. This makes the grep process hard to catch all occurrences, because we need to grep for different keywords. Additionally, notice that [id:1] is not friendly for grep because it may be conflicted with other "id".

We should arrange these fields in a unified way, so that searching a specific keyword will lead and only lead to related log lines.

The following are tasks so that we can pinpoint logs related with different things:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing logging changes in pull requests #5728 and #6055, then identify how the unchecked DMFile, stable layer, delta layer, dm store, and region log fields are currently emitted. Done means defining and applying a consistent grep-friendly format for the remaining areas, with the relevant log lines searchable by their specific fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.