enable_input_metrics: inaccurate values
- Dominant language
- Ruby
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 20
Description
### Describe the bug
We can take metrics of input plugins by setting [enable_input_metrics](https://docs.fluentd.org/deployment/system-config#enable_input_metrics).
However, the value would be inaccurate.
I don't directly confirm it yet, but I have confirmed `@caller_plugin_id` of `EventRouter` has race condition.
* https://github.com/fluent/fluentd/issues/4567#issuecomment-2503380410
So, it would be possible that a wrong `metric_callbacks` is selected.
https://github.com/fluent/fluentd/blob/2d8c9d4b94b1d45e7d4d6e59caae5640a516eabd/lib/fluent/event_router.rb#L96-L102
It would cause wrong metrics calculation.
### To Reproduce
I haven't checked it yet, but the following settings should result in a slight error in the metric values.
```xml
enable_input_metrics
@type monitor_agent
@type sample
tag test.foo
rate 100
@type sample
tag test.bar
rate 100
@type sample
tag test.boo
rate 100
@type null
```
Wait a few minutes and check the metrics.
```bash
curl http://localhost:24220/api/plugins.json | jq
```
```json
{
"plugins": [
{
"plugin_id": "object:d34",
"plugin_category": "input",
"type": "monitor_agent",
"config": {
"@type": "monitor_agent"
},
"output_plugin": false,
"retry_count": null,
"emit_records": 0,
"emit_size": 0
},
{
"plugin_id": "object:d48",
"plugin_category": "input",
"type": "sample",
"config": {
"@type": "sample",
"tag": "test.foo",
"rate": "100"
},
"output_plugin": false,
"retry_count": null,
"emit_records": 43112,
"emit_size": 0
},
{
"plugin_id": "object:d5c",
"plugin_category": "input",
"type": "sample",
"config": {
"@type": "sample",
"tag": "test.bar",
"rate": "100"
},
"output_plugin": false,
"retry_count": null,
"emit_records": 43109,
"emit_size": 0
},
{
"plugin_id": "object:d70",
"plugin_category": "input",
"type": "sample",
"config": {
"@type": "sample",
"tag": "test.boo",
"rate": "100"
},
"output_plugin": false,
"retry_count": null,
"emit_records": 43109,
"emit_size": 0
},
{
"plugin_id": "object:d0c",
"plugin_category": "output",
"type": "null",
"config": {
"@type": "null"
},
"output_plugin": true,
"retry_count": 0,
"emit_records": 129330,
"emit_size": 0,
"emit_count": 129330,
"write_count": 0,
"rollback_count": 0,
"slow_flush_count": 0,
"flush_time_count": 0,
"retry": {}
}
]
}
```
You can confirm a difference in the value of `emit_records` for each `in_sample`.
### Expected behavior
There is no difference in the value of `emit_records` for each `in_sample`.
### Your Environment
```markdown
- Fluentd version: 1.18.0
- Package version:
- Operating system: Ubuntu 20.04.6 LTS (Focal Fossa)
- Kernel version: 5.15.0-124-generic
```
### Your Configuration
```apache
Noted in `To Reproduce`.
```
### Your Error Log
```shell
No error.
```
### Additional context
_No response_
Contributor guide
Research direction
Start with lib/fluent/event_router.rb at lines 96-102 and trace how @caller_plugin_id selects metric_callbacks when enable_input_metrics is enabled. Reproduce the configuration using three sample inputs, monitor_agent, and the plugins API, then compare emit_records across inputs. Done means the input metrics remain accurate and the race condition has regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100