Application not starting or showing state
- Dominant language
- Go
- Stars
- 581
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
Hi! I'm trying to deploy a flink application using the newest operator image (lyft/flinkk8soperator:7fc2230b36ba1d9ee4f78622a56e183abce13be1). The operator deployment is the following:
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: flinkoperator
namespace: test
labels:
app: flinkoperator
spec:
replicas: 1
selector:
matchLabels:
app: flinkoperator
template:
metadata:
labels:
app: flinkoperator
app.kubernetes.io/version: 0.2.0
spec:
serviceAccountName: flinkoperator
volumes:
- name: config-volume
configMap:
name: flink-operator-config
items:
- key: config
path: config.yaml
containers:
- name: flinkoperator-gojson
image: lyft/flinkk8soperator:7fc2230b36ba1d9ee4f78622a56e183abce13be1
command:
- flinkoperator
args:
- --logtostderr
- --config
- /etc/flinkoperator/config*/config.yaml
env:
- name: OPERATOR_NAME
value: flinkk8soperator
imagePullPolicy: IfNotPresent
ports:
- containerPort: 10254
resources:
requests:
memory: "2Gi"
cpu: "4"
limits:
memory: "8G"
cpu: "8"
volumeMounts:
- name: config-volume
mountPath: /etc/flinkoperator/config
```
I have a simple application that only reads the lines from a file and write them into a kafka topic. The application deployment is the following:
```
apiVersion: flink.k8s.io/v1beta1
kind: FlinkApplication
metadata:
name: data-producer
namespace: test
annotations:
labels:
app: data-producer
environment: test
spec:
image: my.registry.com/flink-pipeline:0.2.0
imagePullPolicy: Always
imagePullSecrets:
- name: "gitlab-registry"
flinkConfig:
taskmanager.heap.size: 200
state.backend.fs.checkpointdir: file:///checkpoints/flink/checkpoints
state.checkpoints.dir: file:///checkpoints/flink/externalized-checkpoints
state.savepoints.dir: file:///checkpoints/flink/savepoints
web.upload.dir: /opt/flink
jobManagerConfig:
resources:
requests:
memory: "2000Mi"
cpu: "0.4"
replicas: 1
envConfig:
envFrom:
- configMapRef:
name: flink-common
envFrom:
- configMapRef:
name: data-producer
taskManagerConfig:
taskSlots: 4
resources:
requests:
memory: "2000Mi"
cpu: "0.4"
envConfig:
envFrom:
- configMapRef:
name: flink-common
envFrom:
- configMapRef:
name: data-producer
flinkVersion: "1.9"
jarName: "flink-pipeline.jar"
parallelism: 1
entryClass: "my.package.DataProducerJob"
deleteMode: None
allowNonRestoredState: true
programArgs: >
--my-args args
```
If I run the command `kubectl get flinkapplications.flink.k8s.io -A`, I get the following output:
```
NAMESPACE NAME PHASE CLUSTER HEALTH JOB HEALTH JOB RESTARTS AGE
test ocr-data-producer 17h
```
Both task manager and job manager logs are ok and, at flink's dashboard, the correct task slots number is showing (but no job is running, completed or failed).
PS: I had to change the flinkoperator role, because I was getting a message saying that flinkoperator could not update the *status* of a flinkapplication, so I added this to the role.
**Weird fact**: I started using flink operator v0.3.0 image, as in the example. However, this version is not injecting the rpc address add the `flink-conf.yaml` file, so I have to set the respective environment variable to this. At this version, everything was working. The reason that I have to change the image was due to the **update** lifecycle, which did not work with my rpc workaround.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the operator and FlinkApplication manifests, then inspect operator logs, the FlinkApplication status, and the permissions needed to update status. Compare the newer image with v0.3.0, including RPC configuration and the update lifecycle. Done means the application starts, reports phase and health state, and update handling works without the RPC workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100