elastic / elastic/elastic-agent

[docs-patrol] Document dynamic Kafka topic templates for OTel runtime outputs

Open Beginner friendly
#13,440 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Go
Stars
276
Forks
266
Avg merge
1d 23h
Merged PRs (30d)
312

Description

Recent code changes in the repository have introduced documentation drift. The following change needs corresponding documentation updates.

Changes Requiring Documentation Updates

1. Dynamic Kafka topic templates are now supported in OTel output translation

Commit(s):

  • 94f75d926037d5e81828774905a5f11e9a40ff88 ([beatreceiver] Add support for dynamic kafka topic in OTel)

What changed:

  • internal/pkg/otel/translate/output_kafka.go now supports non-constant Kafka topic expressions (for example %{[data_stream.type]} patterns) by:
    • compiling topic templates,
    • setting topic_from_attribute, and
    • generating a transform processor to populate the topic attribute.
  • The same commit also updates translation wiring in internal/pkg/otel/translate/otelconfig.go so exporter-provided processor config is added to the pipeline.

Documentation impact:

  • User-facing docs do not currently describe this new Kafka topic-template behavior for OTel runtime output translation.
  • Repository docs search found no references for the new behavior (topic_from_attribute, %{[...]} topic templates, or dynamic topic support) under docs/.
  • docs/hybrid-agent-beats-receivers.md documents runtime selection and translation concepts, but does not explain this new Kafka dynamic-topic behavior.

Suggested Actions

  • Update docs/hybrid-agent-beats-receivers.md to document dynamic Kafka topic template support when using OTel runtime translation.
  • Add a concrete configuration example showing %{[...]} topic templates and explain resulting behavior (topic_from_attribute + transform processor).
  • Add/adjust related reference/release-note docs so users discover this capability when configuring Kafka outputs.

[!NOTE]

🔒 Integrity filtering filtered 5 items

Integrity filtering activated and filtered the following items during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

  • resource:search_pull_requests (search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:#unknown (search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • #5280 (search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • #13434 (search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
  • issue:elastic/elastic-agent#unknown (search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

What is this? | From workflow: Docs Patrol

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  • expires on Apr 9, 2026, 2:45 PM UTC

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/hybrid-agent-beats-receivers.md and compare its runtime translation guidance with internal/pkg/otel/translate/output_kafka.go and internal/pkg/otel/translate/otelconfig.go. Add a concrete %{[...] } Kafka topic-template example and explain topic_from_attribute and the generated transform processor. Done means users can discover and understand the dynamic-topic behavior in the relevant documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.