Rework the logging infrastructure.
- Dominant language
- C#
- Stars
- 3.8k
- Forks
- 333
- Avg merge
- 16h 29m
- Merged PRs (30d)
- 116
Description
**How the logging works now**:
Kiota's logging is based on C#'s logging support which uses factories to create loggers with specific categories. The categories are meant to provide a way to add log filters to the infrastructure. Kiota implements a manual `FileLogLoggerProvider` which creates `FileLogLogger` instances on demand. The `FileLogLogger` instances have a file they use to write the logs and close the file on cleanup. The challenge is that you can't have multiple `FileLogLogger`s writing to the same file since each instance locks the file on creation and the next instance can't access it. If we have shared file access, there's the risk of synchronizing accesses to the file. Right now, Kiota works because it only uses 1 logger that's created on startup for all logging.
**Why I want to change it**
Logging is a QoL improvement for language writers and potentially users of Kiota. When implementing a language, implementers might want to log messages to the log file at different stages of generation. Using `Console.WriteLine` would work but is undesirable since the logged messages can't be disabled or easily written to files and other sinks. It will also help us in troubleshooting since users of kiota can provide logs of their runs that might help find bugs faster.
As part of #4208, I would like to provide the logging infrastructure to the language writer classes when needed and allow users to configure logging filters based on components. This isn't a trivial change unless we rework the `FileLogLoggerProvider`s.
**How I want it to behave**
The components that need to log should accept a constructor parameter of type `ILogger` and use that to perform logging throughout the kiota code base. A decision to be made is whether this constructor parameter is optional or required. Whenever a component logs a message, that message should go into the centralized file logger. The expected output is that messages are printed in the order they were submitted.
**How I would implement it**
The 1st thing I would do is move the file management code into the `FileLogLoggerProvider`. This would fix the issue with logger instances failing to access the log file. It would also simplify the `FileLogLogger` type and make it cheap to create.
I would then introduce a concurrent message queue to the `FileLogLoggerProvider` type. The `FileLogLogger` would send any log messages to the provider for processing.
These messages in the queue would be removed from the queue in batches and written to the disk once a specific quota is reached.
When disposing the `FileLogLoggerProvider`, any messages in the queue would be written to the disk one final time.
**Open areas**
Error handling (What happens in the case of an IO error? e.g. If the disk is full when there are still messages in the queue)
Backpressure control (does a thread block or are messages discarded when the queue is full?)
Contributor guide
Research direction
Start by inspecting the FileLogLoggerProvider and FileLogLogger types, then review how logging is currently created and how #4208 would provide loggers to language writer classes. Define the provider-owned file management, ordered queued writes, disposal behavior, and decisions for I/O errors and queue backpressure before implementing. Done means component loggers can write to the centralized file logger without file-access conflicts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- observability, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100