add support for a Revision pre-start hook

In what area(s)?

/area API

Other classifications:
/kind spec

Describe the feature

The notion of an application needing to connect to an external service is a common one. In the past Kube had the ServiceBroker project for this, and today there are other projects out there that are working on a similar set of features. That basic idea is that the environment tries to help the app author by creating a new instance of a service and then provides the connection info (url, creds, ...) via a secret - which is then mounted as a volume, or env vars, into the app.

When the app is already running, the user (or the service broker type of logic) can modify the config of the app (or ksvc) to include the pointer to the secret. However, the problem becomes harder when the user doesn't want the app to come up at all until the secret is attached to it because then it might generate failures while it's waiting for the secret to be attached.

This feature request is NOT to ask for ServiceBroker type of support in Knative. This feature request is for the ability for an admin to config some kind of pre-launch hook for a Revision so that they can add whatever logic they want before the KnServing controller tries to deploy the Revision.

This extensibility point could then be used for other purposes, but our immediate pain point is around Serving Bindings.

Some additional comments/thoughts:

• admission web-hooks (that do some logic, like add the secret) won't work because we need the logic of these hooks to be done asynchronously at times - due to them potentially be long-running tasks
• it's possible that the right place for this might be in Kube, but that's not clear since this might need to be part of some "higher level" logic that delays the creation of the Deployment until the hook's work is done
• the reason Knative might be the right spot for this is because the user-flow that's being considered is that the user adds something like an annotation to the Ksvc to indicate they want some special logic to be done - in this case "connect to my instance of mySQL", e.g. --annotation serving-binding=dugsMySQL. And since we don't want to block the update/creation of the ksvc in etcd, we need some way to pause the serving controller so it waits until this work is done before it generates the Deployment for the Revision. In some ways this is sort of like the opposite of a Kube Finalizer
• doing this above Knative could work, but then it might introduce some interesting challenges. For example, does it then make the "service binding" the top-level object on which the user is working on, rather than making it as an "extra config thing on the ksvc". Or would people to need to create a new CRD to represent ksvc+bindingRequest and then the ksvc is only created when the binding is ready? That, of course, forces a split in the user's workflow when they want to use serviceBindings.
• it'll be interesting to see how this impacts the flow of the controller to introduce a 'pause', but in some ways it might not be that different from someone providing an invalid image name. Except in this case rather than treating it as an error, it just delays the next step in the reconciliation process.

Just looking for some brainstorming on this topic for now...

Start by mapping the KnServing controller's Revision reconciliation flow, especially where it creates the Deployment, and review the proposed asynchronous hook and pause behavior described here. This issue is currently brainstorming rather than implementation-ready; done would require an agreed design for the hook lifecycle, configuration, and completion conditions.