Performance Bottleneck: Slow Log Harvesting in Filebeat filestream Input Under High Throughput Compared to log Input

We are use filebeat version 8.16.3, performing tests with 1500 logs/second for about 15 minutes. We could observe some of the logs that are not getting harvested.
i.e., 1350000 supposed to be sent to respective destination but around 1250000 to 1320000 on repeated tests.

When we analyze the metrics, the logs that are harvested by input type filestream are sent to destination(i.e., logstash). We supsect there is some issue in harvesting the rest of logs.
Similar tests are performed with input type as log and we did not see any loss of logs.

filebeat.yml
```
filebeat.inputs:
- type: filestream
paths:
-
fields:
logplane: "app-logs"
id: my_filestream_id_1
ignore_inactive: since_last_start
close.reader.after_interval: "5m"
prospector.scanner.check_interval: 1s
fields_under_root: true
processors:
- dissect:
when:
regexp:
log.file.path: '^/var/log/pods/[^/]+_[^/]+_[^/]+/[^/]+/[^/]+$'
tokenizer: "/var/log/pods/%{namespace}_%{pod.name}_%{pod.uid}/%{container.name}/%{log.file.name}"
field: "log.file.path"
target_prefix: "kubernetes"
ignore_failure: true
- drop_fields:
fields:
- "kubernetes.log.file.name"
ignore_missing: true
output.logstash:
hosts: "logstash:5044"
ssl.certificate_authorities:
ssl.certificate:
ssl.key:
ssl.verification_mode: "full"
ssl.renegotiation: "freely"
ssl.supported_protocols: ["TLSv1.2", "TLSv1.3"]
ssl.cipher_suites: []
worker: 2
ttl: 30
queue.mem:
flush.timeout: 1s
filebeat.registry.flush: 5s
logging.level: "debug"
logging.selectors: ["harvester", "registrar"]
logging.metrics.enabled: true
http.enabled: true
http.host: localhost
http.port: 5066
```
For confirmed bugs, please report:
- Version: 8.16.3
- Operating System: K8S, SUSE based container.
- Discuss Forum URL: https://discuss.elastic.co/t/loss-of-logs-occurred-while-migrating-from-log-input-to-filestream-input/370532
- Steps to Reproduce:

Start by reproducing the Filebeat 8.16.3 test with the provided filestream configuration and compare it with the log input under the same load. Inspect the harvester, registrar, and enabled metrics while checking the expected 1,350,000 events against the destination count. Done means identifying the cause of the discrepancy or documenting a confirmed, repeatable failure with complete reproduction steps.

Written by the indexing model from the issue text.