NLog / NLog/NLog

Allow AsyncWrapper to flush in batches instead of one giant flush

Open
#3,856 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

file-target
Dominant language
C#
Stars
6.5k
Forks
1.4k
Avg merge
9h 17m
Merged PRs (30d)
24

Description

Have an application that writes several GByte data using NLog as part of startup-rutine.

After startup-rutine has completed, and just about to go online, then it performs a blocking garbage collection.

It could be nice if one could flush NLog buffers before starting the garbage collection. The problem is that the async-target might have a lot of data pending at this point.

If one calls NLog LogManager.Flush() then it will take the entire async-queue-buffer and write in one operation.

Lets say 200.000 items are still pending and each of them is 1000 bytes. Then it becomes close to 2 gigabyte which is the upper limit for a MemoryStream. Calling NLog LogManager.Flush() will then either lead to application crashing with out-of-memory exception, or discarding data because of stream-is-too-long.

It could be nice if AsyncWrapper had a MaxBatchSize (nullable) that could be configured instead of BatchSize. And it would also have effect during flush-operations. Instead of writing 200.000 in one operation, then it would do 200 write-operations of MaxBatchSize=1000

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating AsyncWrapper and the LogManager.Flush() path, then trace how pending queue items are passed to the target during a flush. Verify the behavior with a large pending queue and confirm that a configured maximum batch size limits each flush operation without discarding data.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
observability
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.