elastic / elastic/integrations
[Kafka Consumer (OpenTelemetry)]: Add azure_auth extension support
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Kafka Consumer (OpenTelemetry) [packages/kafka_input_otel]
### Dataset Name
kafkareceiver
### Integration Version
0.1.1
### Agent Version
9.4.4
### OS Version and Architecture
N/A
### User Goal
Starting with Elastic-Agent 9.3.6 and 9.4.3 (https://github.com/elastic/elastic-agent/pull/14534), Elastic-Agent includes the Azure Auth extension that can be used to authenticate via Azure Entra ID.
However, this extension is currently not usable in Fleet because it should be used by the relevant integrations.
We would like to achieve the following configuration:
```
extensions:
azure_auth:
service_principal:
client_id: "${env:CLIENT_ID}"
tenant_id: "tenant-id"
client_secret: "${env:CLIENT_SECRET}"
scopes:
- "api://confluent-cloud/.default"
receivers:
kafka:
brokers:
- "${env:GATEWAY_KAFKA_BROKER}"
protocol_version: "2.1.0"
logs:
topics:
- "${env:KAFKA_LOGS_TOPIC}"
metrics:
topics:
- "${env:KAFKA_METRICS_TOPIC}"
traces:
topics:
- "${env:KAFKA_TRACES_TOPIC}"
group_id: "${env:KAFKA_CONSUMER_GROUP}"
tls:
insecure: false
insecure_skip_verify: false
auth:
sasl:
mechanism: OAUTHBEARER
version: 0
oauthbearer_token_source: azure_auth
...
service:
extensions:
- azure_auth
pipelines:
...
```
### Existing Features
The current feature does not have support for azure_auth extension.
### What did you see?
N/A
### Anything else?
N/A
Contributor guide
Research direction
Start in packages/kafka_input_otel and trace how the Kafka consumer currently handles authentication configuration. Compare that path with the requested azure_auth extension and service wiring; done means the shown configuration is accepted and the consumer can use Azure Entra ID authentication.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, kafka
- Domain
- authentication, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100