CI is slow

We should investigate what it takes to speed up CI. I have some suggestions, but I'm sure more can be done. Feel free to add anything I forgot

- [x] Benchmarks on windows runner is very slow compared to linux. We should consider decreasing number of samples needed on windows
- [ ] The same `wit tests` run twice, once for GUEST=c, and once for GUEST=rust, but in this case the tests are identical, so no need to rerun.
- [ ] We should be able to trim down some dev-dependencies to improve compile times using cargo features
- [ ] -Can we cache rust artifacts in github actions cache to improve compile times?