[MVC] General routing improvements

Work for routing in 6.0.
* Review the list of issues labeled area-routing

There are three areas of improvement here:
* Link generation compatibility improvements beetween old routing and new routing.
* Memory usage improvements for the DFAMatcher we use by default.
* Link generation equivalent for dynamic route transformers.

#### Link generation compat improvements
Link generation works slightly differently with endpoint routing than old routing and this surprises customers. We want to see what work we can do here to "smooth" that out to ease migration out of old routing.

#### Memory usage improvements
We had some customers report large memory usage by routing in some situations. There are two things that we can do here:
* Trim tree when there is a parameter with a constraint by checking candidate against the constraint.
* Trim tree when there is a complex parameter that will never be matched by a candidate.

#### Link generation equivalent for route transformers
We have support for dynamically matching an endpoint, however we don't have the same functionality on the "other direction". This is one of the things you can do with old routing that you can't with the current routing system and that prevents us from moving away completely from it.