Please, make CI caching more tolerant

In #2503 the caching workflow for the most important part (GHC 9.2) - would fail. Because `cabal v2-build all` would not work there.

Think about, would it be reasonable to cache all OR main+test OR main. That would allow acquiring relevant partial cache, which would allow a faster CI loop to solve problems. But that also would happen when `freeze` is not possible (since `build all` has the exact same requirement) & when `freeze` is not possible - under certain conditions we do not have an effective way to detect when to save new cache.

At the same time we really can not expand cache more pretty much due to pool limit & for the same reason, we reaching a limit on expanding the key matrix more.