rtk-ai / rtk-ai/rtk

sqlfluff filter is verbose against plain `sqlfluff lint` on small corpora

Open
#3,920 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:cli enhancement help wanted priority:low
Dominant language
Rust
Stars
81.1k
Forks
5.1k
Avg merge
4d 21h
Merged PRs (30d)
35

Description

rtk sqlfluff lint condenses heavily against the JSON it requests, but sqlfluff's own text output is already terse, so on small runs the filter's framing costs more than it saves relative to simply running sqlfluff lint.

Measured against real sqlfluff (4.3.0 by @TheGlitching, reproduced on 3.2.5):

corpus plain sqlfluff lint rtk sqlfluff lint vs plain
2 files, 8 violations 1,100 1,377 −25.2%
4 files, 34 violations 3,471 3,787 −9.1%
4 files, 18 violations 2,627 2,638 −0.4%
51 files, 73 violations 10,849 5,673 +47.7%
51 files (reproduced) 18,764 6,452 +65.6%

Against the baseline rtk actually tracks — the JSON it asks sqlfluff for — the filter is at 85.8–96.2%, well clear of the 20% floor in CONTRIBUTING.md. So this is not a floor violation; it is that below roughly 50 violations the Violations: section restates every violation with a full path prefix, and everything above it (Top rules, Top files) becomes pure addition.

ruff does not have this problem because its native output is far more verbose (66.8% on a comparable corpus).

Two reshapes were measured on the 4-file corpus in #253:

  • group the detail lines under a per-file header — 3,618 bytes, +16.5%
  • move each rule's description into Top rules, leaving line:col CODE in the detail — 1,757 bytes, +59.5%

The second is the bigger win and arguably reads better, since Top rules currently shows a code with no indication of what it means. It was deliberately left out of #253 because the section's shape is inherited from ruff_cmd, and changing it in one filter would make the two diverge.

That is the actual decision here, and it is why this is an issue rather than a change in #253: is the Top rules / detail split worth reshaping across the filter family, or is the per-file header the right local fix? Whoever takes this should decide it for ruff_cmd and sqlfluff_cmd together.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read CONTRIBUTING.md and compare the existing ruff_cmd and sqlfluff_cmd output shapes, using the measurements in this issue and #253 as context. Decide whether the shared Top rules/detail reshaping or per-file headers is the better approach for both filters, then verify the chosen output against small and large corpora.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.