Test in ci unwanted changes over hackage index GPD check
- Dominant language
- Haskell
- Stars
- 1.7k
- Forks
- 750
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 28
Description
* From https://github.com/haskell/cabal/pull/8055#issuecomment-1073681055
* We can get how any Cabal version affects GPD checks with `cabal run Cabal-tests:hackage-tests -- check`:
```
D:\ws\haskell\cabal>cabal run Cabal-tests:hackage-tests -- check
Up to date
Reading index from: D:\cabal\packages\hackage.haskell.org\01-index.tar
157946 files processed
42067 files have lexer/parser warnings
115687 files have check warnings
1118 build impossible
7980 build warning
51952 build dist suspicious
36949 build dist suspicious warning
17688 build dist inexcusable
```
* Afaics those numbers are not checked in ci and it would have revealed regressions as the caused one by #7616
* We could leverage it:
* Regenerating the stats in a known file using the last Cabal master version *and a fixed hackage index*. Automatically via makefile or using another tool.
* Making `Cabal-tests:hackage-tests -- check` generate stats with HEAD *and the same hackage index* and failing if they differ in any way with the saved one (so we would check if we are being too restrictive or too loose)
* Forcing the developer update numbers on purpose to make the test pass (or revise their code)
Contributor guide
Assessment
This issue has not been assessed yet.