realm / realm/SwiftLint

Running lint without any code changes is still slow (30s vs 40s without cache)

Open
#5,207 16 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug discussion
Dominant language
Swift
Stars
19.7k
Forks
2.3k
Avg merge
1d 1h
Merged PRs (30d)
11

Description

New Issue Checklist
Describe the bug

I noticed that our SwiftLint build phase was taking a really long time to run in our project, even when no code had changed. Digging further, on my machine with our project, it was taking 30s to run without any changes. What is even stranger is that it was only taking 10s more to run without any cache at all.

Using the time profiler in instruments, I found that most of that time was taken up looking for excluded files in Glob.expandGlobstar. It appears that every excluded pattern causes SwiftLint to retrieve a recursive list of subpaths. The documentation for FileManager.subpathsOfDirectory(atPath:) calls this out as a potential performance issue.

Complete output when running SwiftLint, including the stack trace and command used
$ swiftlint lint --quiet
Environment
  • SwiftLint version: 0.52.2 (also tested latest master)
  • Installation method used: mint (also tested from source and running the command directly)
  • Paste your configuration file:

Here is an abbreviated list of our excluded patterns:

  - ManualFrameworks
  - vendor
  - fastlane
  - SourceryOutput
  - Generated
  - templates
  - "**/.build"
  - "**/*.generated.swift"
  - "**/LocalizedStrings.swift"
  • Are you using nested configurations? No
  • Which Xcode version are you using (check xcodebuild -version)? 14.3.1 and 15.0 beta 8

I have a proof of concept that matches files based on regular expressions (converted from the glob syntax) and uses a directory enumerator to test for matches. This has a few advantages: First, the directory hierarchy can be scanned once regardless of how many include and exclude patterns are used. Second, if a directly is excluded (or not included) it can be skipped entirely no need to even scan the directory contents. We can also re-use the attributes from the enumerator to test if it is a file or folder and avoid calling fileExists(atPath:,isDirectory:).

With this change, I'm seeing the fully cached time go from 30s to 10s.

This seems to be a fairly common technique. Some scripting languages even have the glob to regex converter built in. It also appears to be what SwiftFormat does.

Before I clean this up and submit a PR, I wanted to make sure I wasn't missing anything.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Glob.expandGlobstar and reproduce the reported case with swiftlint lint --quiet, comparing cached and uncached runs. Review the linked proof of concept and directory-enumerator approach; done means excluded glob patterns are handled without repeated recursive scans while preserving lint behavior and improving the fully cached runtime.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
devtools, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.