huggingface / huggingface/transformers-mlinter
Decide a per-rule cutoff burn-down: the cutoff currently hides 3,867 violations
- Dominant language
- Python
- Stars
- 6
- Forks
- 12
- Avg merge
- 10h 28m
- Merged PRs (30d)
- 19
Description
Reported by developers
> Was there any reason for the cutoff date because it might be too gracious and hiding too much across models
>
> For example autodocstrings should be at top level models but based on older models we skip too graciously
Opening this to answer the question with numbers and to make the burn-down a deliberate decision per rule rather than a standing one-date policy.
## Why the cutoff exists
`cutoff_date` grandfathers models contributed before a convention was introduced, so a rule does not have to ship an allowlist of every model that predates it. Without it the alternative is a ~500-entry `allowlist_models` on most rules.
## What it hides today
Two full runs over a `transformers` checkout of 2026-08-25 (510 models), all rules enabled: **429 findings as shipped, 4,296 with every `cutoff_date` set to `1900-01-01`.** So the cutoff currently hides **3,867** violations. Per rule:
| rule | hidden | rule | hidden | rule | hidden |
|---|---|---|---|---|---|
| TRF041 | 1744 | TRF037 | 118 | TRF024 | 25 |
| TRF045 | 735 | TRF023 | 118 | TRF028 | 17 |
| TRF057 | 393 | TRF036 | 115 | TRF056 | 17 |
| TRF027 | 149 | TRF034 | 96 | TRF030 | 10 |
| TRF029 | 141 | TRF019 | 66 | TRF026 | 9 |
| | | TRF035 | 64 | TRF042 | 5 |
| | | TRF033 | 38 | TRF025 | 4 |
(TRF041 #52, TRF029 #53, TRF034 #54 and TRF035 #55 have separate false-positive issues open; their numbers here are before those fixes. The inheritance defect behind the "parent is grandfathered" complaint is #56.)
## Proposal
`cutoff_date` is already per rule, so this does not need a mechanism change — it needs a decision per rule. For rules whose fix is mechanical and reviewable in bulk, run a one-time cleanup PR in `transformers` and then delete that rule's `cutoff_date`:
- **TRF057** — `@auto_docstring` on public classes and methods, 393 findings. This is the one the feedback asks for, and adding the decorator is mechanical.
- **TRF028** — license headers, 17 findings. Trivially mechanical.
- **TRF045** — `output_attentions` / `output_hidden_states` / `return_dict` in `forward`, 735 findings. Larger and behavioural, so a candidate for staging rather than one sweep.
Rules whose remaining findings are judgement calls (TRF041, TRF036, TRF037) should keep the cutoff until their false-positive issues land, and be re-measured afterwards.
## Context worth recording
`transformers`' repo-local `utils/rules.toml` has been accumulating **model-wide** allowlist entries to unblock new-model PRs — TRF018 has 16 repo-only entries (`gemma4`, `kimi_k25`, `olmo3`, `radio`, `sam3_lite_text`, `sam3_tracker`, `voxtral_realtime`, ...), TRF034 six, TRF019 six, TRF041 three, TRF016 two.
That is strictly worse than the cutoff: a cutoff exempts a model for code written before the convention existed, whereas an allowlist entry mutes the rule for that model forever, including code written tomorrow. It is also a good measure of how much friction the current defaults are causing model adders.
Contributor guide
Research direction
Start by reviewing the measurements and per-rule proposal in this issue, then inspect transformers' repo-local utils/rules.toml and the cutoff_date settings it records. Compare each rule's hidden findings and related issues before deciding which rules need a cleanup PR, which should retain the cutoff, and what recorded per-rule outcome constitutes done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100