fluent / fluent/fluentd

Metrics (buffer_queue_length and buffer_total_queued_size) not available with detached processes

Open
#855 4 comments 0 reactions 0 assignees View on GitHub
enhancement v0.14
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

When `detach_process` is used in BufferedOutput, children have its own buffer in my understanding. So, `monitor_agent` can't find `buffer_queue_length` and `buffer_total_queued_size` for that plugin.

Do you have any plan to support buffer metrics for `DetachProcessMixin` and `DetachMultiProcessMixin` by `monitor_agent`?

One of my suggestion is here:

```
{
"plugin_id": "object:3fea11e14868",
"plugin_category": "output",
"type": "kinesis_streams",
"config": ...,
"output_plugin": true,
"detach_processes": [
{
"buffer_queue_length": 0,
"buffer_total_queued_size": 0
}
],
"retry_count": 0
}
```

See also https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/45

Contributor guide

Open the contributing guide

Research direction

Start by reading DetachProcessMixin and DetachMultiProcessMixin alongside monitor_agent's handling of buffer_queue_length and buffer_total_queued_size. Determine how detached child buffers could be represented in the suggested detach_processes structure; done means monitor_agent exposes those metrics for detached processes with appropriate coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.