aws / aws/aws-logging-dotnet

Provide mechanism to flush background queue of log messages

Open
#211 0 comments 1 reaction 0 assignees View on GitHub
feature-request module/logging p2 queued
Dominant language
C#
Stars
307
Forks
135
Avg merge
1d 10h
Merged PRs (30d)
4

Description

### Describe the feature

Developers that use this library in Lambda have the issue that log messages queued in the background queue don't get processed before the Lambda function is complete and the Lambda compute container is frozen. If no more Lambda events are sent to the Lambda compute container then those log messages are lost.

To ensure all messages are sent before the Lambda function finish processing an event provide a method from AWS.Logger.Core that developers can call to flush the queue.

This is the issue that inspired me to write this feature request but is far from the first to callout the limitations of these libraries in Lambda.
https://github.com/aws/aws-logging-dotnet/issues/204

### Use Case

Developers want to use these logging libraries in Lambda because they want their log messages to go to their own log groups. Currently we don't recommend using these libraries in Lambda due to the background processing nature but there is clearly a demand for doing this based other GitHub issues. We should provide them a flushing mechanism to make it work. It does mean that developers will have to wait finishing their Lambda event being processed till the messages are pushed to CloudWatch Logs.

### Proposed Solution

For every logging output in each of our supported plugins there is an instance of `AWSLoggerCore` created to manage the background processing queue. Most cases there will only be one instance of `AWSLoggerCore` but it is possible there could be multiple instances. For example developers could configure info messages to go to one target and error messages to go to another target. Consumers of any of our plugin libraries have no direct access to the instance of `AWSLoggerCore`.

We need to maintain a registry of every instance created. Basically whenever the constructor of `AWSLoggerCore` is called add it to some static registry. Whenever the `Close` method is called remove the instance from the registry. Then somewhere in core package provide a public static method that triggers a flush on all registered instances of `AWSLoggerCore`. This flush should take in a timeout so we don't get stuck flush indefinitely.

### Other Information

_No response_

### Acknowledgements

- [X] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### AWS .NET SDK and/or Package version used

Latest

### Targeted .NET Platform

.NET Core 3.1 and .NET 6

### Operating System and version

Linux

Contributor guide

Open the contributing guide

Research direction

Start in the core package by locating AWSLoggerCore's constructor and Close method, then review how supported plugins create and own instances. Done means a public static operation can flush every registered instance with a timeout, while Close removes instances from the registry; the payload names no tests or specific files.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, csharp
Domain
backend, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.