concourse / concourse/concourse-chart

concourse_workers_containers custom metric issue

Open
#200 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Mustache
Stars
160
Forks
189
PR merge metrics
No merged PRs in 30d

Description

Hi! We're using the customMetric concourse_workers_containers as defined in the chart's example, and we're using prometheus adapter along with it.
The issue is that that concourse is calling the custom metric with the app label as concourse-worker, while the metric in Prometheus is being saved while associated with the concourse-web app. For example:

concourse_workers_containers{container="concourse-web",endpoint="prometheus",instance="<instance>",job="concourse-web-prometheus",namespace="<namespace>",platform="linux",pod="concourse-web-5d47f5c978-hmmqf",service="concourse-web-prometheus",worker="concourse-worker-6465dbfb5c-87qww"}

The query being done to the prometheus-adapter is something along these lines:

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/:namespace/pods/*/concourse_workers_containers_avg?labelSelector=app=concourse-worker,release=concourse"

This yields no result as the metric is associated with the concourse-web app. Therefore, changing the query to something along these lines retrieves the correct result:

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/:namespace/pods/*/concourse_workers_containers_avg?labelSelector=app=concourse-web,release=concourse"

Is there any way in the chart to associate the concourse_workers_containers metric with the concourse-worker app so it can be successfully fetched?

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the chart's customMetric example and the Prometheus Adapter labelSelector query shown in the issue. Trace how the concourse_workers_containers metric is associated with app labels, then verify that a query using app=concourse-worker retrieves the metric without breaking the existing concourse-web query.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, prometheus
Domain
devops, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.