Make the output of the node communications trace logging more usable
- Dominant language
- C#
- Stars
- 5.5k
- Forks
- 1.5k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 141
Description
### Summary
Today the node communication logs are hard to work with - they are relatively unstructured text, and each process writes to a distinct file. To get a view of the whole system, you have to do a lot of manual inspection and collation across a number of disparate files. As a result, it's difficult to correlate data across the node processes + scheduler node for each build.
### Background and Motivation
We need a way to view/interact with this data that is still readable at-a-glance, but it more amenable to tooling.
### Proposed Feature
We should change the communications logging infrastructure to emit structured logs efficiently. The existing APIs should:
* take string formats instead of already-rendered strings for performance
* accept message templates with named 'holes'
* emit the messages with structured metadata:
* PID
* Thread Id
* timestamp
* the rendered message
* key-value pairs for the data 'rendered' into the message
* the structured data should be emitted as atomic lines to a single unified log file
* the format of this log file should be easily-queryable - json lines, csv, etc.
* log query tools natively understand these formats and make it easy to query/join the lines to get "all of the logs for the build that started a TIME_X"
### Alternative Designs
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.