Support using NetworkPolicy with Routes

The Problem

Today, Routes include a cluster-internal URL which is implemented by creating a headless Kubernetes Service with the same name as the Route, and with endpoints selected (subsetted) from the set of HTTP routers supplied by the networking plugin. Callers are expected (as with externally-visible routes) to include a Host: HTTP header which indicates which Knative Route they are dialing; callers without the appropriate HTTP/1.1 header will be dropped as not matching any rules.

Unfortunately, this approach means that traffic to Knative Routes in different namespaces cannot be distinguished at an L4 level by most CNIs, and it means that callers MUST use a correct HTTP Host header to reach the Knative Route. The former (L4 traffic is identical at the CNI level) means that developers implementing Knative services cannot use Kubernetes NetworkPolicy to restrict access to their applications (unless using a service mesh or L7-aware CNI). This also violates the principle of least surprise for security and network policy admins, and takes away a simple and handy tool. The latter problem with Host headers mostly affects HTTP gateway implementations which may simply pass along the existing Host header unless explicitly programmed to rewrite it.

See this drawing for a summary of the current situation.

The Solution

This Feature Track document proposes a solution, which has been implemented in net-kourier.

https://github.com/knative-sandbox/net-kourier/pull/852 implements per-namespace port allocation on Envoy and changes to the per-Route Kubernetes Service management such that each namespace is allocated a unique port on the Envoy proxy, and each Kubernetes Service representing an internal Route uses independent service Endpoints which use the namespace-specific port rather than a universal port. (This could also be implemented with route-specific rather than namespace-specific ports.)

Scalability

While scalabality may appear to be an issue, there should be at least 8k ports available on each envoy instance, which is likely to result in at least 3*8k Kubernetes Services being created given current Knative usage (one for the Route, and two for each Revision). This is likely to hit other Kubernetes scaling limits before it hits ports-per-Envoy limits.