temporalio / temporalio/temporal

Expand configuration for Elastic to support serverless

Open
#6,253 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
23.2k
Forks
1.9k
Avg merge
2d 8h
Merged PRs (30d)
228

Description

Is your feature request related to a problem? Please describe.

We were looking to use Elastic's serverless option for our visibility store, but to connect we would need to set an Authorization header, which currently doesn't appear to be supported by the config for the ES client: https://github.com/temporalio/temporal/blob/66ab754e7e8d60421678457df1914fa7ddae9dc3/common/persistence/visibility/store/elasticsearch/client/config.go#L45-L60

Describe the solution you'd like

I'd like to be able to add a configuration like the following:

elasticsearch:
  # ...
  headers:
    Authorization: "ApiKey {{ .Env.TEMPORAL_ES_VISIBIILITY_STORE_PASSWORD }}"

Describe alternatives you've considered

Another option would be to have this kind of a format:

elasticsearch:
  # ...

  auth:
    basic:
      username: "..."
      password: "..."
    # or
    elastic:
      apiKey: "..."

But it feels far less flexible and a bit more complicated to implement.

Other than that, I can't really think of any alternative, besides having to set up a proxy that adds the header. Ideally, we avoid the operational overhead of another proxy and just do it straight on the client.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with common/persistence/visibility/store/elasticsearch/client/config.go at the configuration section linked in the issue, then trace how those settings reach the Elasticsearch client. Add support for configuring request headers and verify that the Authorization header is accepted and used when connecting to the visibility store.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
backend, databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.