apache / apache/pinot

Reduce the amount of trace info

Open
#9,523 2 comments 0 reactions 1 assignee Claimed by @soupam05 View on GitHub
good first issue help wanted
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Currently the default tracer will print an entry per block (up to 10K docs). For a query hitting 1B docs, it will log 100K entries, which is not useful for debugging. It can even hang the controller UI when sending a query with trace on.

We should reduce the tracing info by merging the same entry type into one entry for each segment and just print the total time per entry type.

Other things to consider:
- Should we merge the same entry type across segments? Each query can still hit thousands of segment per server
- We can consider keeping a global cross segments aggregated trace info, then keep 10 segment level aggregated trace if the query hits too many segments

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.