realm / realm/SwiftLint

Excluded files impact the performance of swiftlint

Open
#5,018 11 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement integration
Dominant language
Swift
Stars
19.7k
Forks
2.3k
Avg merge
1d 1h
Merged PRs (30d)
11

Description

New Issue Checklist
Describe the bug

The number of files present in the working tree of swiftlint has a huge impact on its performance even if most of these files are ignored. Here's a concrete example/repro:

  • Create a temporary directory with just one file to lint:
mkdir /tmp/swiftlint_tests && cd /tmp/swiftlint_tests
touch test.swift
  • Use hyperfine to measure the execution time of swiftlint:
$ hyperfine --warmup 1 'swiftlint'
➜  swiftlint_tests hyperfine --warmup 1 'swiftlint'
  Time (mean ± σ):      61.4 ms ±   0.9 ms    [User: 52.2 ms, System: 7.1 ms]
  Range (min … max):    59.9 ms …  65.4 ms    46 runs
  • Create a subdirectory with a lot of swift files in it, create a linter config to exclude these files:
mkdir .build
for i in {1..10000}
do
  touch .build/$i.swift
done

echo "excluded:\n  - '**/.build'" > .swiftlint.yml
  • Make sure that these ignored files are actually ignored:
$ swiftlint
Linting Swift files in current working directory
Linting 'test.swift' (1/1)
Done linting! Found 0 violations, 0 serious in 1 file.
  • Benchmark swiftlint again:
➜  swiftlint_tests hyperfine --warmup 1 'swiftlint'
  Time (mean ± σ):     378.7 ms ±   6.1 ms    [User: 229.2 ms, System: 377.6 ms]
  Range (min … max):   368.2 ms … 389.6 ms    10 runs

The execution time is 6x what it is when these ignored files don't exist. Using the --use-alternative-excluding flag brings this to 5x but it'd still be nice to completely ignore these file. Not using a glob pattern in the config improves things by another 1x.

It looks like the code is traversing the entire file tree in a few places (e.g. in the glob implementation), we could maybe make it use a recursive approach and stop when it encounters a subdirectory that is ignored (e.g. in my example it'd not look at the content of the .build directory at all).

Environment
  • SwiftLint version (run swiftlint version to be sure)?
    From source

  • Installation method used (Homebrew, CocoaPods, building from source, etc)?
    Source

  • Paste your configuration file:
    See example

  • Are you using nested configurations?
    No

  • Which Xcode version are you using (check xcodebuild -version)?

$ xcodebuild -version
Xcode 14.3
Build version 14E222b

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing SwiftLint's file-tree traversal and glob implementation, including the path used by --use-alternative-excluding. Reproduce the hyperfine comparison with one Swift file and 10,000 excluded files, then verify that excluded directories are not fully traversed and that linting output remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
cli, performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.