Change the value of httpGetTimeOut in Flink Client
- Dominant language
- Go
- Stars
- 581
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/lyft/flinkk8soperator/blob/b5eb19657df91782e51028019e89a7c1badd30d1/pkg/controller/flink/client/api.go#L39
It looks like the submitting of our application takes more time rather than 5 seconds. it's cause to loop the submitting process and eventually to fail it.
The output of flink operator looks like the following:
```
"ClusterStarting:Handling state for application"
"ClusterStarting:Flink cluster has started successfully"
"Savepointing:Handling state for application"
"SubmittingJob:Handling state for application"
"SubmittingJob:No job found for the application"
"SubmittingJob:Logged Warning event: JobSubmissionFailed: Failed to submit job to cluster for deploy 4e1550d8: SubmitJob call failed with status FAILED and message '': Post http://jobmanager:8081/jars/job.jar/run: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
"SubmittingJob:Conflict while updating status"
"SubmittingJob:Failed to reconcile resource application: Operation cannot be fulfilled on flinkapplications.flink.k8s.io \"application\": the object has been modified; please apply your changes to the latest version and try again"
"SubmittingJob:Handling state for application"
"SubmittingJob:No job found for the application"
"SubmittingJob:Logged Warning event: JobSubmissionFailed: Failed to submit job to cluster for deploy 4e1550d8: SubmitJob call failed with status FAILED and message '': Post http://jobmanager:8081/jars/job.jar/run: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
"SubmittingJob:Failed to reconcile resource application: SubmitJob call failed with status FAILED and message '': Post http://jobmanager:8081/jars/job.jar/run: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
"SubmittingJob:Handling state for application"
"SubmittingJob:Conflict while updating status"
"SubmittingJob:Failed to reconcile resource application: Operation cannot be fulfilled on flinkapplications.flink.k8s.io \"application\": the object has been modified; please apply your changes to the latest version and try again"
"SubmittingJob:Handling state for application"
"SubmittingJob:Failed to reconcile resource application: found multiple active jobs for application"
"SubmittingJob:Handle state skipped for application, lastSeenError UnknownMethod call failed with status FAILED and message '': found multiple active jobs for application"
```
Contributor guide
No contributing guide indexed for this repository
Research direction
The timeout is defined in pkg/controller/flink/client/api.go at line 39; start there and trace its use in the job-submission request. Increase the timeout from 5 seconds so submissions no longer cancel while awaiting headers and trigger repeated attempts; verify the reported reconciliation failure no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100