elastic / elastic/beats

JWT(OAuth2?) Support - Elasticsearch output

Open
#46,662 2 comments 0 reactions 0 assignees View on GitHub
Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 1h
Merged PRs (30d)
370

Description

**Describe the enhancement:**

Similar to the request in #22603, it would be great if we could use JWT authentication in an elasticsearch output, given that [JWT auth is supported with elasticsearch](https://www.elastic.co/docs/deploy-manage/users-roles/cluster-or-deployment-auth/jwt).

While it is possible to set a JWT as an authentication header as @andrewkroh describes in [this comment](https://github.com/elastic/beats/issues/22603#issuecomment-3298942867), what I'm thinking is a step further back in the authentication flow(I think its described as the `access_token` type on the elasticsearch side).

Perhaps I'm phrasing it incorrectly here, but what I would like is the ability for a beats output client to use a client id, client secret, and a scope to reach out to an OAuth2 *provider*, retrieve a JWT from there, and use that for future requests against the elasticsearch defined output. I have an example of what I imagine _could_ work as a config below.

This also assumes that when said JWT is about to expire, this output would automatically attempt to gather a new token from that provider, since that information will be encoded in the JWT itself.

This last part is a huge plus, since its pretty common to have pretty short-lived JWT's, on the order of a day or a few hours. Whereas the client-id and client-secret are much longer lived(at least in my experience) on the order of at least a few months, if not longer.

**Describe a specific use case for the enhancement or feature:**

This would allow us to use an external identity provider to manage credentials on our beats agents rather than TLS client certs as we use today.

**Example configuration**

```yaml
output.elasticsearch:
hosts: ["10.45.3.2:9220", "10.45.3.1:9230"]
oauth2:
endpoint: https://identity-provider-endpoint.enterprise
client_id: some-app-identitifer
client_secret: some-secret
# similar to multiple values in a header, multiple scopes would be space-separated
scope: "someapp:elaticsearch"
protocol: https
path: /elasticsearch
```

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Elasticsearch output configuration and its existing authentication flow; the issue names no files or tests. Define what is needed for endpoint, client credentials, scope, token acquisition, expiry refresh, and authenticated requests, then confirm the behavior against the example configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
authentication, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.