Improve CI tests run time

### Description

Tests (and CI in particular) take too long to run IMHO.
Some avenues to consider beyond some work on added with #1519
1. Selectively run only tests when needed based on an analysis of code changes (complex but likely worth it)
2. Break the main repo in more repos such that we only run tests on each sub-repo when these repos are changed. (essentially leading to 1. with the caveat that we cannot have proper integration tests running at all time for instance if a change in copyright detection has some impact on license detection or license detection impacts package detection )
3. Run less tests combos in the CI matrix, for instance drop running certain Python version/ OS combos that are not likely to be super useful
4. Drop Python 3.6 support?
5. Improve boot times: there is over 1 minute spent to ./configure and this could be faster. On macOS updating homebrew can take forever too. May be switching to container tests on Linux would speed things up. Combining the tests of several Python versions on a single OS in one task may be more efficient and remove some boot time.
6. Run fewer tests on each OS/Python combos: we need only a basic smoke test to ensure that things are working on every combo. This could be made faster by running fewer tests including for the pip installation tests.