mado and mdl disagree on 36 of the 52 acceptance fixtures
- 主要語言
- Rust
- 星號
- 407
- 分支
- 12
- 平均合併
- 5 小時 45 分鐘
- 30 天內合併 PR
- 30
描述
The acceptance comparison is not clean, and nothing recorded that. Raised while
fixing #396, which repaired the setup script that prepares this corpus.
## What the comparison shows
`scripts/acceptance/test.sh` runs `mdl` and `mado check --output-format=mdl`
over the same fixtures, each under one fixed config (`scripts/acceptance/.mdlrc`
and the repository's `mado.toml`), and writes `tmp/mdl.txt` and `tmp/mado.txt`.
Diffing those two files on a corpus prepared by `setup.sh`:
- **36 of 52 fixtures differ.**
- 103 findings mado reports and mdl does not.
- 30 findings mdl reports and mado does not.
Both files use the same `path:line: MDxxx text` form and absolute paths, so
these are real disagreements, not formatting.
| mado only | | mdl only | |
| --- | --- | --- | --- |
| MD007 | 40 | MD031 | 20 |
| MD041 | 31 | MD007 | 3 |
| MD003 | 6 | MD032 | 2 |
| MD025 | 5 | MD022 | 2 |
| MD031 | 4 | MD028 | 1 |
| MD022 | 3 | MD024 | 1 |
| MD040, MD038, MD020, MD006, MD004, MD002 | 2 each | MD013 | 1 |
| MD028, MD014 | 1 each | | |
`MD007`, `MD041` and `MD031` are all listed in `.mdlrc`, so this is not a
question of one tool having a rule the other was not asked for. MD031 is the
interesting one: it appears on **both** sides, so the two tools are finding it
in different places rather than one of them missing it.
## Why it is worth a look now
The corpus these numbers come from is what `setup.sh` is supposed to produce:
the fixtures upstream pairs with an `X_style.rb` are set aside, because those
are written against a rule set of their own that `test.sh` cannot apply. So
this is the intended comparison, and it is the one that disagrees.
Two things to separate before reading much into the totals:
- Whether `.mdlrc` and `mado.toml` really do ask for the same thing. They are
written in different formats and were not derived from one another.
- Which differences are mado bugs, which are mdl bugs, and which are documented
intentional divergences. Nothing in the repository currently says.
## Note on how this went unnoticed
`setup.sh`'s exclusion loop has never run in the committed script — see #396 —
so anyone cloning fresh compares all 88 fixtures instead of these 52, on a
corpus that includes documents designed for other rule sets. The numbers above
come from a checkout where the exclusion had been applied by hand before the
script was committed.
`test.sh` also has no `diff` and no assertion: it writes two files and stops.
So there has never been anything to notice, which is #400's subject from a
different angle.
貢獻指南
評估
這個 Issue 還沒有評估資料。