kubeflow / kubeflow/spark-operator
[Question] Pluggable support for customized substate and events consuming and handling?
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 1.5k
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 13
Description
### What question do you want to ask?
- [ x ] ✋ I have searched the open/closed issues and my issue is not listed.
### Additional context
Hi community,
I am wondering if there is any pluggable way to extend the current operator behavior to cover more complicated use cases, including capturing and handling customized events, potentially allow user to define customized `ApplicationStateType` to make it more native. And it would be nicer if we can write those into the CRD and log them somewhere for trouble shooting later.
One of the use cases would be we often found our job got stuck in `SUBMITTED` state due to various reasons. Depending on user implementation, a lot of things can happen during that stage: scheduler scheduling, node provisioning, driver image downloading, driver pod init container running, etc. It will be much user friendly to allow user plug in logic to capture and handle events generated by different components and report them with a customized state in CRD.
### Have the same question?
Give it a 👍 We prioritize the question with most 👍
Contributor guide
Research direction
No file or test is named in the question. Start by tracing the operator’s existing ApplicationStateType and event-handling entry points, then examine how states, events, and troubleshooting information are represented in the CRD and logs. Done means an agreed, implementable extension path with clear scope and validation criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- backend-api-design, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100