sqlfluff filter is verbose against plain `sqlfluff lint` on small corpora
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 81.1k
- Forks
- 5.1k
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 35
Description
rtk sqlfluff lint condenses heavily against the JSON it requests, but sqlfluff's own text output is already terse, so on small runs the filter's framing costs more than it saves relative to simply running sqlfluff lint.
Measured against real sqlfluff (4.3.0 by @TheGlitching, reproduced on 3.2.5):
| corpus | plain sqlfluff lint |
rtk sqlfluff lint |
vs plain |
|---|---|---|---|
| 2 files, 8 violations | 1,100 | 1,377 | −25.2% |
| 4 files, 34 violations | 3,471 | 3,787 | −9.1% |
| 4 files, 18 violations | 2,627 | 2,638 | −0.4% |
| 51 files, 73 violations | 10,849 | 5,673 | +47.7% |
| 51 files (reproduced) | 18,764 | 6,452 | +65.6% |
Against the baseline rtk actually tracks — the JSON it asks sqlfluff for — the filter is at 85.8–96.2%, well clear of the 20% floor in CONTRIBUTING.md. So this is not a floor violation; it is that below roughly 50 violations the Violations: section restates every violation with a full path prefix, and everything above it (Top rules, Top files) becomes pure addition.
ruff does not have this problem because its native output is far more verbose (66.8% on a comparable corpus).
Two reshapes were measured on the 4-file corpus in #253:
- group the detail lines under a per-file header — 3,618 bytes, +16.5%
- move each rule's description into
Top rules, leavingline:col CODEin the detail — 1,757 bytes, +59.5%
The second is the bigger win and arguably reads better, since Top rules currently shows a code with no indication of what it means. It was deliberately left out of #253 because the section's shape is inherited from ruff_cmd, and changing it in one filter would make the two diverge.
That is the actual decision here, and it is why this is an issue rather than a change in #253: is the Top rules / detail split worth reshaping across the filter family, or is the per-file header the right local fix? Whoever takes this should decide it for ruff_cmd and sqlfluff_cmd together.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read CONTRIBUTING.md and compare the existing ruff_cmd and sqlfluff_cmd output shapes, using the measurements in this issue and #253 as context. Decide whether the shared Top rules/detail reshaping or per-file headers is the better approach for both filters, then verify the chosen output against small and large corpora.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100