Speeding up CI

CI is currently pretty slow. In particular, if there are several PRs in quick succession, we'll run out of CI capacity. A reasonable medium-term goal might to get CI builds finish in 10 minutes in typical cases. I'm not sure if that is realistic, however.

Here are some ideas about how to make things better:

• Drop appveyor (it seems redundant now) [done]
• Run selected slow tests only after a PR has been merged
  • Alternatively, only run them once per PR iteration, and run on all platforms and Python versions after merge
• If a PR only touches mypyc, don't run non-compiled mypy tests
• Remove some slow tests from CI if they don't bring much value
• Run more tests in incremental mode
• If a PR only touches docs, don't run all tests
• Test fewer platform / Python version combinations (possibly only in PR builds)
• Speed up individual tests somehow
• Somehow get more CI capacity

Can anybody suggest more things that could help, or which of the above things would be worth pursuing first?

The issue names no files, tests, or CI entry points and presents several alternatives rather than a selected task. Start by identifying the CI configuration and measuring a typical build; done would require an agreed optimization with evidence that CI is faster.