realm / realm/SwiftLint

Add coverage statistics

Open
#5,906 0 comments 0 reactions 0 assignees View on GitHub

@mildm8nnered is already working on this.

Since Dec 22, 2024.

  • #5907 by @mildm8nnered — open
enhancement
Dominant language
Swift
Stars
19.7k
Forks
2.3k
Avg merge
1d 1h
Merged PRs (30d)
11

Description

New Issue Checklist
Feature or Enhancement Proposal

It would be nice if SwiftLint could report "coverage" statistics. In the context of a linter, what does coverage mean?

  • If every rule was applied to every line of code, then coverage would be 1, or 100%.
  • If no rules are applied to any lines of code, then coverage would be zero.
  • If half the rules are applied to all of the code, then coverage should be .5 or 50%.
  • If all the rules are applied to half of the code, then coverage should also be .5

So we can define coverage as

sum of the number_of_rules_applied_to_each_line / (total_number_of_lines * number_of_rules)

number_of_rules can be defined in two ways:

The number of enabled rules ("Enabled rules coverage")
The total number of rules, enabled or not ("All rules coverage")

The first metric ("Enabled rules coverage") provides a measure of how much enabled rules are being suppressed with commands like swiftlint:disable. Generally we'd expect this coverage figure to be very high, but if some malicious actor was to go around writing // swiftlint:disable all at the top of every file, it would drop to zero. For SwiftLint itself, coverage on this metric is .995.

The second metric ("All rules coverage") provides a measure what coverage you're getting, compared to the coverage you would get if all rules were enabled. A configuration that enabled only one rule would have very low coverage - as some rules are contradictory, in a real deployment this figure would rarely be 1. For SwiftLint itself, coverage on this metric is .861.

The corresponding figures for Periphery are 1, and 0.36, as not that many rules are enabled in Periphery's configuration.

A few other definitions:

number of lines of code - as SwiftLint can flag up violations in blank lines or comments as well as code, it makes more sense to count every line in every file, rather than lines which actually contain code per se.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the coverage definitions and proposed enabled-rules and all-rules metrics in issue #5906. Review linked pull request #5907 to see the work already underway, then verify that the reported statistics match the formulas and line-counting rules described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.