Add support to chunk testing large sets of test projects and run them in parallel

Today for things like the `net - core` pipelines we are testing a large number of projects in the same job which is only growing over time. Not only does that cause the pipeline to run slower and slower (currently about 60mins) but it makes retrying failures painful. To help with this we need to figure out a way to dynamically generate a test matrix that will allow us to run a subset of the full project list on independent agents.

This will allow us to increase the overall time and allow retrying to be much more efficient.

This chunking will be interesting for all the languages but the .NET core pipeline and soon the core mgmt pipeline will be needing this support to be able to maintain a reasonable testing time for PR validation.