Evaluate controller-runtime

Question

I've been doing lots of "copy-paste" work when developing #1121 .

It is always debatable that how much abstractions we should do, anyway, DRY(Don't repeat yourself) is somehow opposite to KISS(Keep it simple and stupid). But I start to feel that the boilerplate in our codebase is slightly too much.

We could evaluate some frameworks to see if they can help us simplify the code and logic, actually there is only one possible choice: kubernetes-sigs/controller-runtime.

I did not mention kubebuilder and operator-sdk because I think skeleton tool means little to us (we are not going to adopt a skeleton or code generation tool, at least not in this issue). and the framework they are leveraging is exactly controller-runtime.

#1291 has imported controller-runtime as a dependency to use its generic client. But there are more useful abstractions in controller-runtime that requires minor refactoring to the codebase, we don't have enough effort to land such change in v1.1.0, nor could we do it without thorough discussion and evaluating.

Open this issue for further discussion and evaluating.