stackabletech / stackabletech/nifi-operator
Allow loading python processors from a git repository in NiFi 2.0
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 48
- Forks
- 12
- Avg merge
- 11h 26m
- Merged PRs (30d)
- 10
Description
NiFi 2.0 adds support for python processors that can be used in flows.
These can be loaded by simply providing them in a directory on the pod and configuring NiFi to look into this directory.
NiFi also supports hot reloading of these processors out of the box, so this would be a perfect use case for a git-sync sidecar container, like we are using for Airflow already.
### Tasks
- [ ] Allow specifying a gitsync sidecar container (potentially with multiple repos)
- [ ] Mount repositories into the pods
- [ ] Configure NiFi to load processors from these mounts
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing Airflow git-sync sidecar approach and the operator's NiFi pod and configuration handling. Done means repositories can be synchronized, mounted into NiFi pods, and configured as processor locations, including the stated possibility of multiple repositories.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, kubernetes, python, rust
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100