lyft / lyft/flinkk8soperator

Can't use ConfigMap for /opt/flink/conf because entrypoint tries to write to it

Open
#168 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
581
Forks
153
PR merge metrics
No merged PRs in 30d

Description

It looks like you are relying on the ENTRYPOINT script to append the contents of OPERATOR_FLINK_CONFIG env var to `/opt/flink/conf/flink-conf.yaml`. Unfortunately, this makes it impossible to use a ConfigMap mounted as a volume to provide a default `flink-conf.yaml` and `log4j.properties`, because ConfigMap volumes are intentionally read-only. I can't see a way to make this happen.

Alternatively, you could provide a way for the Operator to control default values for checkpoint folders etc. For example to ensure that state is always stored to a specific place (eg. S3 bucket) with the Job name as a component in the path (job name as one of the folder names). Or, to provide the Zookeeper quorum info for that environment. Generally, you could allow the operator to provide (templated) default flink-conf.yaml. An appropriate `log4j.properties` can be included in a Docker base image. But it would be nice for the operator to provide job-specific config.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the ENTRYPOINT behavior that appends OPERATOR_FLINK_CONFIG to /opt/flink/conf/flink-conf.yaml and examine how ConfigMap volumes are mounted. Clarify whether the goal is to avoid writes to a mounted configuration directory or to add operator-controlled, templated defaults; done should support the chosen configuration path without the read-only volume failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.