stackabletech / stackabletech/nifi-operator
Idea: Start ReportingTaskJob only once the NifiCluster is ready
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 48
- Forks
- 12
- Avg merge
- 11h 26m
- Merged PRs (30d)
- 10
Description
There is a reporting task started immediately: https://github.com/stackabletech/nifi-operator/blob/db33e5e32c55975369e15a063ae21ef728498c2c/rust/operator-binary/src/controller.rs#L277
I want to suggest to follow a pattern we've implemented in Superset: Add a status to the NifiCluster and let the operator listen to it. Only start the job once the cluster is ready.
At the moment the job is started and remains in a CrashLoopBackOff, until the Cluster is ready. This doens't look good and can also cover up actual problems with the job (because this CrashLoopBackoff is 'normal' for us). It would also make the task finish quicker, because at the moment once the cluster is ready, the task is still in a retry-waiting-period. Only once this has passed the job finishes. This can easily save 30 seconds of time.
Edit: since k8s 1.24 Jobs can be started in the suspended state, so the Job object could already be created, but only started once the cluster is up.
Edit: Alternatively the Operator could do the HTTP request, and we could drop the Job altogether
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in rust/operator-binary/src/controller.rs around line 277, then compare the readiness-listening pattern referenced from Superset. Decide which proposed approach is in scope—waiting for NifiCluster readiness, suspending the Job, or replacing it with an HTTP request—and verify that the reporting task no longer enters a normal CrashLoopBackOff before the cluster is ready.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, rust
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100