Excluded files impact the performance of swiftlint
Nobody has claimed this yet.
- Dominant language
- Swift
- Stars
- 19.7k
- Forks
- 2.3k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 11
Description
New Issue Checklist
- Updated SwiftLint to the latest version => I built it from source
- I searched for existing GitHub issues
Describe the bug
The number of files present in the working tree of swiftlint has a huge impact on its performance even if most of these files are ignored. Here's a concrete example/repro:
- Create a temporary directory with just one file to lint:
mkdir /tmp/swiftlint_tests && cd /tmp/swiftlint_tests
touch test.swift
- Use hyperfine to measure the execution time of swiftlint:
$ hyperfine --warmup 1 'swiftlint'
➜ swiftlint_tests hyperfine --warmup 1 'swiftlint'
Time (mean ± σ): 61.4 ms ± 0.9 ms [User: 52.2 ms, System: 7.1 ms]
Range (min … max): 59.9 ms … 65.4 ms 46 runs
- Create a subdirectory with a lot of swift files in it, create a linter config to exclude these files:
mkdir .build
for i in {1..10000}
do
touch .build/$i.swift
done
echo "excluded:\n - '**/.build'" > .swiftlint.yml
- Make sure that these ignored files are actually ignored:
$ swiftlint
Linting Swift files in current working directory
Linting 'test.swift' (1/1)
Done linting! Found 0 violations, 0 serious in 1 file.
- Benchmark swiftlint again:
➜ swiftlint_tests hyperfine --warmup 1 'swiftlint'
Time (mean ± σ): 378.7 ms ± 6.1 ms [User: 229.2 ms, System: 377.6 ms]
Range (min … max): 368.2 ms … 389.6 ms 10 runs
The execution time is 6x what it is when these ignored files don't exist. Using the --use-alternative-excluding flag brings this to 5x but it'd still be nice to completely ignore these file. Not using a glob pattern in the config improves things by another 1x.
It looks like the code is traversing the entire file tree in a few places (e.g. in the glob implementation), we could maybe make it use a recursive approach and stop when it encounters a subdirectory that is ignored (e.g. in my example it'd not look at the content of the .build directory at all).
Environment
-
SwiftLint version (run
swiftlint versionto be sure)?
From source -
Installation method used (Homebrew, CocoaPods, building from source, etc)?
Source -
Paste your configuration file:
See example -
Are you using nested configurations?
No -
Which Xcode version are you using (check
xcodebuild -version)?
$ xcodebuild -version
Xcode 14.3
Build version 14E222b
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing SwiftLint's file-tree traversal and glob implementation, including the path used by --use-alternative-excluding. Reproduce the hyperfine comparison with one Swift file and 10,000 excluded files, then verify that excluded directories are not fully traversed and that linting output remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- cli, performance, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100