lyft / lyft/flinkk8soperator

Configuring Prometheus metric reporter

Open
#137 21 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
581
Forks
153
PR merge metrics
No merged PRs in 30d

Description

Good day.
We use Flink Prometheus metric reporter (https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter) in conjunction with Prometheus Operator ServiceMonitor (https://github.com/coreos/prometheus-operator/blob/master/Documentation/user-guides/getting-started.md#related-resources) to collect Flink cluster metrics. But when we tried to swithc to flinkk8soperator we encountered some difficulties.

To collect metrics we need to expose metric reporter port 9249 on both services- JobManager and TaskManager, but FlinkApplication configuration does not allow configuring container ports, and flinkk8soperator creates a Service only for JobManager.

Can we configure FlinkApplication to collect metrics without changing the source code of flinkk8soperator?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the FlinkApplication configuration and how the operator creates JobManager and TaskManager services. Compare the required port 9249 exposure with Prometheus Operator ServiceMonitor expectations. Done means metrics can be collected for both Flink components through configuration, without changing the operator source.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, prometheus
Domain
infrastructure, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.