elastic / elastic/beats

[azure-eventhub] Add support for Workload Identity

Open
#48,835 1 comment 0 reactions 1 assignee Claimed by @zmoog View on GitHub
input:azure-eventhub Team:obs-ds-hosted-services
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 2h
Merged PRs (30d)
364

Description

## Context

The azure-eventhub input originally used connection strings as the only authentication method. Since introducing processor v2 with the modern Event Hubs SDK, we now support multiple authentication options through the `azidentity` module.

**Workload Identity** is the recommended authentication method for Kubernetes workloads on Azure Kubernetes Service (AKS) and other Kubernetes distributions. It replaces the deprecated pod-managed identity approach and enables secure, secret-free authentication using OpenID Connect (OIDC) federation with Microsoft Entra ID.

Key characteristics:
- **Zero secrets management** - No credentials stored in cluster or configuration
- **Kubernetes-native** - Uses Service Account Token Volume Projection
- **Multi-cloud support** - Works on AKS, Azure Arc-enabled clusters, and other Kubernetes distributions
- **Industry standard** - Currently used by Azure Monitor for Prometheus remote write authentication
- **Enhanced security** - Short-lived tokens with automatic rotation

The azure-eventhub input uses the Azure Event Hubs SDK based on the `azidentity` module, which supports `WorkloadIdentityCredential` out of the box.

## Goal

TBD

## Impact

By adding Workload Identity support, we enable:

- Enhanced Security - Eliminates credential storage in Kubernetes clusters, reducing attack surface and meeting compliance requirements for secret-free deployments
- Simplified Operations - No certificate rotation, key management, or secret synchronization across multiple clusters
- Broader Adoption - Aligns with Microsoft's recommended approach for AKS workloads, making the azure-eventhub input the standard choice for Kubernetes-based log collection
- Multi-cloud Flexibility - Supports Azure Arc-enabled Kubernetes clusters running on-premises or in other clouds, expanding deployment options
- Better Developer Experience - Follows the same pattern as Azure Monitor's Prometheus remote write, providing consistency across Azure observability tools

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.