elastic / elastic/docs-content
[REQUEST]: Add "Elastic Agent to Kafka to Kafka ES Sink to Elasticsearch" architecture
- Dominant language
- No language data
- Stars
- 47
- Forks
- 261
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 116
Description
### Description
We have `Elastic Docs ›Elastic Ingest Reference Architectures [8.16] ›Elastic Agent to Elasticsearch with Kafka as middleware message queue
Elastic Agent to Logstash to Kafka to Kafka ES Sink to Elasticsearch: Kafka as middleware message queue` at . However, Logstash is not necessary between Elastic Agent as it can ship directly to Kafka. Please add this to this to the documentation.
Also please add the configuration for Kafka ES Sink that will pass through the collected data types (e.g logs, metrics, etc.) to the correct data stream and ingest pipelines. This is a critical piece of information for understanding how to set this all up.
Also, could add the link to the Sink documentation on page as well.

### Resources
- https://www.elastic.co/guide/en/ingest/current/agent-kafka-essink.html
- https://www.elastic.co/blog/index-data-elasticsearch-confluent-cloud
- https://www.elastic.co/blog/elastic-agent-output-kafka-data-collection-streaming
- https://www.elastic.co/blog/generative-ai-applications-confluent-cloud-apache-flink-elasticsearch
- https://docs.confluent.io/cloud/current/connectors/cc-elasticsearch-service-sink.html
- https://docs.confluent.io/cloud/current/connectors/limits.html#elasticsearch-service-sink-limits
- https://www.elastic.co/blog/elastic-agent-send-process-any-data-anywhere
- https://www.elastic.co/guide/en/fleet/current/kafka-output-settings.html
### Collaboration
Other (please describe below in the Point of contact section).
### Point of contact.
**Main contact:** @
**Stakeholders:**
- @a03nikki (I can help review the documentation as I have a user who is looking for this specifically)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.