elastic / elastic/integrations
[Azure Logs]: Allow Custom Azure Blob Storage Endpoint in Elastic Agent Azure Integration for Sovereign/DoD Clouds (IL6)
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
### Integration Name
Azure Logs [azure]
### Dataset Name
azure.*
### Integration Version
1.28.7
### Agent Version
9.1.4
### OS Version and Architecture
Windows and Linux
### User Goal
In sovereign and DoD Azure environments (e.g., Impact Level 6/IL6), the Azure Blob Storage endpoint differs from the default public cloud endpoint. For example, in IL6, the correct endpoint is blob.core.microsoft.cloud, but the Elastic Agent Azure integration currently infers or hardcodes the blob endpoint as blob.azure.microsoft.scloud when the Resource Manager endpoint is set to https://management.azure.microsoft.scloud.
This results in integration failures when attempting to use the Azure Event Hub integration for log ingestion, as the checkpointing mechanism cannot reach the correct blob storage endpoint.
Because of this, the Elastic Agent is not able to leverage the checkpointing capability and scale out the number of the Elastic Agents consuming logs from Azure Event Hubs. The goal of this feature request is to allow Elastic Agents to correctly leverage Azure Blob for checkpointing.
### Existing Features
Current Behavior:
The integration only allows configuration of the Resource Manager endpoint.
The blob endpoint is not configurable and is incorrectly derived for IL6 and similar sovereign clouds.
Expected Behavior:
The integration should allow users to explicitly specify the Azure Blob Storage endpoint (e.g., blob.core.microsoft.cloud) independently of the Resource Manager endpoint.
This would enable successful operation in all Azure sovereign and DoD environments.
Impact:
Unable to use Elastic Agent Azure integrations in IL6 and similar environments due to checkpointing failures.
Compliance and operational requirements for US government and DoD customers cannot be met.
Workarounds Attempted:
No documented or supported method to override the blob endpoint in the integration.
The Elasticsearch snapshot repository supports azure.client.default.endpoint, but this is not available in Elastic Agent integrations.
### What did you see?
Steps to Reproduce:
Configure the Azure integration in Elastic Agent for an IL6 environment.
Set the Resource Manager endpoint to https://management.azure.microsoft.scloud.
Observe that the integration attempts to use blob.azure.microsoft.scloud instead of blob.core.microsoft.cloud for checkpointing, resulting in failure.
### Anything else?
Request:
Please add support for specifying a custom Azure Blob Storage endpoint in the Elastic Agent Azure integration, to enable compatibility with all Azure sovereign and DoD clouds.
Contributor guide
Assessment
This issue has not been assessed yet.