elastic / elastic/elastic-agent

Feature Request: Deterministic Priority for Internal Fleet Server URL and Elasticsearch Output URL

Open
#11,639 0 comments 1 reaction 0 assignees View on GitHub
Team:Elastic-Agent-Control-Plane Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
275
Forks
264
Avg merge
1d 20h
Merged PRs (30d)
303

Description

## Summary

Elastic Agent currently accepts multiple Fleet Server URLs and multiple Elasticsearch output URLs. However, the agent lacks deterministic prioritisation logic. When both internal (on-prem) and external (public) endpoints are reachable, the agent may select either. This results in unpredictable routing for both **Fleet Server communication** and **data output ingest**.

This feature request proposes introducing **strict, ordered priority** for both Fleet URLs and Elasticsearch output URLs. The agent should always prefer the internal URL and only use external URLs as a fallback when the primary becomes unreachable.

## Problem Statement

In hybrid deployments—roaming laptops, intermittent VPN connections, partially disconnected clients—agents often have configurations that include:

* Internal Fleet Server URL
* Public Fleet Server fallback
* Internal Elasticsearch output
* Public Elasticsearch output fallback

Today, when the device has simultaneous reachability (e.g. inside VPN **and** with public Internet), the agent:

* May enroll or communicate with the public Fleet Server even if an internal server is available.
* May send logs or security telemetry to the wrong Elasticsearch cluster.
* Creates compliance risks by routing telemetry outside expected boundaries.
* Produces inconsistent ingest behavior, complicating SOC and platform operations.

This unpredictable selection is unacceptable in regulated industries such as healthcare, critical infrastructure, and finance.

## Requested Enhancement

Add deterministic priority and failover logic for **both** Fleet Server URLs and Elasticsearch output URLs.

### Required behavior

1. Agent always tries the **primary/internal URL** first.
2. Secondary/public URLs are only used when the primary is unreachable.
3. When the primary becomes reachable again, the agent automatically returns to it.
4. Priority must be deterministic and admin-controlled.
5. A configuration flag such as `prefer_primary: true` should activate this behavior.

### Applies to:

* Fleet Server URLs (`fleet.server.urls`)
* Elasticsearch output URLs (`outputs.elasticsearch.hosts`)

## Proposed Configuration Example

```yaml
fleet:
server:
urls:
- https://fleet-internal.company.local
- https://fleet-public.company.com
prefer_primary: true

outputs:
default:
type: elasticsearch
hosts:
- https://es-internal.company.local:9200
- https://es-public.company.com:9200
prefer_primary: true
```

## Benefits

* Predictable routing for all Elastic Agent components.
* Guarantees on-prem ingest whenever internal connectivity is present.
* Eliminates governance/data residency violations caused by misrouted telemetry.
* Reduces operational complexity in SOC and platform operations.
* Aligns Elastic Agent with deterministic failover patterns used in other EDR/endpoint products.

## Impact

This feature would significantly improve Elastic Agent behavior in enterprise, hybrid, and regulated environments. It provides explicit and transparent control over where security and observability data flows, without relying on unpredictable reachability heuristics.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how fleet.server.urls and outputs.elasticsearch.hosts are parsed and how endpoint selection currently handles reachability. Define and validate deterministic primary-first failover, recovery to the primary, and the prefer_primary configuration for both endpoint types.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.