minScale treatment races reachability check

In what area(s)?

/area autoscale

What version of Knative?

All recent at least.

Expected Behavior

autoscaling.knative.dev/minScale holds the respective scale of a revision after it becomes ready.

Actual Behavior

We sometimes scale below the mark only to then scale up again.

Example failing test: https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_serving/11217/pull-knative-serving-istio-stable-no-mesh/1383957220156248064

Steps to Reproduce the Problem

This is a race with the labeler component, which labels all revisions targeted by a Route to be "Reachable". If a revision is unreachable, we ignore the minScale annotation, since nothing can hit the respective revision anyway. This allows more efficient resource usage for minScaled revisions.

Possible solutions

1. Make the Unreachable state more consistent with routing state

In the Revision reconciler, we translate a certain RoutingState (as given by the labeler) to a given Reachability state on the KPA spec (which then informs the minScale decision). We already have a pending state for Revisions that have not been reconciled by the labeler yet. That means, if we'd base strictly off of that RoutingState, this race should be fixed.

However: As it stands, it's possible that a revision is never touched by the labeler, for example if Configuration and Route are created independently. Such a Revision stays in pending forever and as such, would never scale down. That may or may not be fine.

2. Allow for the race

We can allow for the race and adjust the respective test to actually wait for route state to kick in before starting to assert the number of pods.

Personally I'd love to make 1 fly, to have a reliable and consistent RoutingState that can be used as a useful signal for things. We just need to be aware to not change semantics in an unintended way, as Garbage Collection is relying on this mechanism too. Arguably, the semantics we're after are actually quite close to those of Garbage Collection so that might be a good thing!

Start with the Revision reconciler, the labeler component, and the RoutingState-to-Reachability translation in the KPA spec. Reproduce the minScale race using the linked failing test, then compare the two proposed approaches, including the effects on revisions never touched by the labeler and on Garbage Collection. Done means minScale behavior is consistent without unintended routing or garbage-collection changes.