opensearch-project / opensearch-project/data-prepper
[advise] winlogbai 7.12.1 + dataprepper + opensearch implementation
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Hello Team,
I need some advise to implement a new environments with winlogbait agent (7.12.1) sending event logs to opensearch throught dataprepper.
i used this settings:
logging.to_files: true
logging.files:
path: C:\ProgramData\winlogbeat\Logs
logging.level: info
output.elasticsearch:
hosts: ["localhost:9200"]
enabled: false
ssl.certificate: "/etc/pki/client/cert.pem"
ssl.key: "/etc/pki/client/cert.key"
output.logstash:
hosts: ["192.168.107.1:7104"]
enabled: true
ssl.enabled: true
rootca is installed in ROOTCA OS envirronement
but i have this issue in logs:
2022-12-15T17:20:25.594+0100 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: write tcp 192.168.0.66:59140->192.168.107.1:7104: wsasend: Une connexion existante a dû être fermée par l’hôte distant.
2022-12-15T17:20:25.594+0100 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(async(tcp://192.168.107.1:7104))
2022-12-15T17:20:25.594+0100 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
also i tried to create index in opensearch by using winlogbeat.template.json by this method:
curl --insecure -s -H 'Content-Type: application/json' -XPUT https://opensearch-node1:9200/_index_template/winlogbeat-7.12.1 -u 'admin:XXXXX' --data-binary "@winlogbeat.template.json";
but i have this issue :
{"error":{"root_cause":[{"type":"invalid_index_template_exception","reason":"index_template [winlogbeat-8.5.3] invalid, cause [Validation Failed: 1: unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings;2: expected [index.lifecycle.name] to be private but it was not;]"}],"type":"invalid_index_template_exception","reason":"index_template [winlogbeat-8.5.3] invalid, cause [Validation Failed: 1: unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings;2: expected [index.lifecycle.name] to be private but it was not;]"},"status":400}
thanks for your help
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the posted Winlogbeat configuration, the publisher_pipeline_output log, and winlogbeat.template.json. Trace the Data Prepper endpoint at 192.168.107.1:7104 and review the reported index.lifecycle.name validation error; done means identifying compatible settings for the Winlogbeat-to-Data Prepper-to-OpenSearch path.
Written by the indexing model from the issue text.
Assessment
- Domain
- data, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100