dotnet / dotnet/msbuild

Make the output of the node communications trace logging more usable

Open
#13,240 0 comments 0 reactions 0 assignees View on GitHub
Area: Debuggability internal-team-onboarding triaged
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 8h
Merged PRs (30d)
141

Description

### Summary

Today the node communication logs are hard to work with - they are relatively unstructured text, and each process writes to a distinct file. To get a view of the whole system, you have to do a lot of manual inspection and collation across a number of disparate files. As a result, it's difficult to correlate data across the node processes + scheduler node for each build.

### Background and Motivation

We need a way to view/interact with this data that is still readable at-a-glance, but it more amenable to tooling.

### Proposed Feature

We should change the communications logging infrastructure to emit structured logs efficiently. The existing APIs should:

* take string formats instead of already-rendered strings for performance
* accept message templates with named 'holes'
* emit the messages with structured metadata:
* PID
* Thread Id
* timestamp
* the rendered message
* key-value pairs for the data 'rendered' into the message
* the structured data should be emitted as atomic lines to a single unified log file
* the format of this log file should be easily-queryable - json lines, csv, etc.
* log query tools natively understand these formats and make it easy to query/join the lines to get "all of the logs for the build that started a TIME_X"

### Alternative Designs

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.