stackabletech / stackabletech/nifi-operator

Idea: Start ReportingTaskJob only once the NifiCluster is ready

Open
#293 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
48
Forks
12
Avg merge
11h 26m
Merged PRs (30d)
10

Description

There is a reporting task started immediately: https://github.com/stackabletech/nifi-operator/blob/db33e5e32c55975369e15a063ae21ef728498c2c/rust/operator-binary/src/controller.rs#L277

I want to suggest to follow a pattern we've implemented in Superset: Add a status to the NifiCluster and let the operator listen to it. Only start the job once the cluster is ready.

At the moment the job is started and remains in a CrashLoopBackOff, until the Cluster is ready. This doens't look good and can also cover up actual problems with the job (because this CrashLoopBackoff is 'normal' for us). It would also make the task finish quicker, because at the moment once the cluster is ready, the task is still in a retry-waiting-period. Only once this has passed the job finishes. This can easily save 30 seconds of time.

Edit: since k8s 1.24 Jobs can be started in the suspended state, so the Job object could already be created, but only started once the cluster is up.

Edit: Alternatively the Operator could do the HTTP request, and we could drop the Job altogether

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rust/operator-binary/src/controller.rs around line 277, then compare the readiness-listening pattern referenced from Superset. Decide which proposed approach is in scope—waiting for NifiCluster readiness, suspending the Job, or replacing it with an HTTP request—and verify that the reporting task no longer enters a normal CrashLoopBackOff before the cluster is ready.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, rust
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.