linkerd / linkerd/linkerd2

Support custom authentication with external prometheus instance

Open
#7,038 9 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/viz help wanted needs/design
Dominant language
Go
Stars
11.5k
Forks
1.4k
Avg merge
3d 22h
Merged PRs (30d)
53

Description

## Feature Request

### Background

We're running linkerd in a relatively large cluster, and the integrated Prometheus instance is pretty much useless unless you give it a ton of memory on a dedicated worker node. Moreover, why would I run my own Prometheus when cloud providers can do the heavy lifting.

### What problem are you trying to solve?

Bring your own Prometheus instance that requires some sort of authentication, such as token-based (static), basic auth.

```
GET /prometheus/api/v1/query HTTP/1.1
Host: external.prometheus.com
Authorization: Bearer supersecuretoken
```

### How should the problem be solved?

`viz` plugin should have a way to accept arbitrary HTTP headers

Add a new flag to the `metrics-api` entrypoint, something like `prometheusAuthHeaders`, and it is expecting a string array. For example, `--prometheusAuthHeaders key1=val1,key2=val2` or `--prometheusAuthHeaders key1=val1 --prometheusAuthHeaders key2=val2`

https://github.com/linkerd/linkerd2/blob/451602e28da51a0db4ff053e33cefe9f1eda9a17/viz/metrics-api/cmd/main.go#L21-L29

In the helm chart, we can just add a new value and map them to the metrics-api deployment command arguments.

```yml
prometheusAuthHeaders:
Authorization: "Bearer supersecuretoken"
More: "Custom Headers"
```

To include the auth headers, we will need to change how the prom API client is initialized.

This can be implemented using the `RoundTrippers` from the prometheus API client somewhere around here https://github.com/linkerd/linkerd2/blob/451602e28da51a0db4ff053e33cefe9f1eda9a17/viz/metrics-api/cmd/main.go#L49-L55

Learn more about how the round trippers work https://github.com/prometheus/client_golang/pull/817

### Any alternatives you've considered?

Is there another way to solve this problem that isn't as good a solution?

No.

### How would users interact with this feature?

If you can, explain how users will be able to use this. Maybe some sample CLI
output?

See the previous section.

### Notes

I am open to work on it. It would be nice if the linkerd team can provide some guidance on the testing strategy.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with viz/metrics-api/cmd/main.go, especially the metrics-api entrypoint and Prometheus client initialization referenced in the issue, then review the Helm chart values and deployment arguments. Check the Prometheus client RoundTrippers guidance and existing tests before defining coverage; done means external Prometheus requests can receive configured arbitrary headers through the CLI and Helm configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, helm, kubernetes, prometheus
Domain
api, devops, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.