JamieMason / JamieMason/syncpack
feat: prune ignored directories during file discovery
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
### Description
Syncpack's file discovery walks the entire workspace tree, visiting every directory except `node_modules` and `.git`. It does not prune directories excluded by `.gitignore`, nor does it limit traversal to the directories implied by the workspace globs, even when explicit `--source` flags are passed.
In large monorepos this can dominate total runtime. In our workspace (~630 projects, pnpm workspaces, 120 package.json files), with a 1.4M-file jest cache directory at the workspace root (gitignored): syncpack lint took ~63 seconds. After removing that directory: ~4.5 seconds. In a minimal repro (3 packages), adding 40,000 gitignored junk files took syncpack lint from 0.05s to ~13.5s.
Additionally, on corporate machines like mine, endpoint protection (Microsoft Defender) intercepts the walker's per-file accesses, at ~0.3ms per not-yet-cached file. When there are more files on the walk, this adds up over time, making the walk even slower. Since build/test caches regenerate files constantly, the cost recurs on every run.
Reproduced on 14.3.0 and 15.3.2. The 15.2.0 fix for #334 corrected which files appear in results, but the walk itself still visits everything. Syncpack seems the right place to fix this because no CLI flag can currently avoid the traversal (`--source` filters results, not the walk).
### Suggested Solution
No CLI or config surface change strictly needed, ideally discovery just gets faster.
- Prune traversal using `.gitignore` (e.g. the ignore crate's WalkBuilder, which handles nested gitignores natively), the same way ripgrep does
- Only descend into directories that can match the workspace globs / source patterns (e.g. `packages: ["apps/*", "libs/*"]` never requires entering `./.jestcache/**` or `./dist/**`).
If implicit gitignore behavior is undesirable, an opt-in config would also solve it:
```json
{
"ignorePaths": [".jestcache/**", "dist/**", "coverage/**"]
}
```
Expected outcome: `syncpack lint` output unchanged, but runtime proportional to the number of candidate package files rather than total files in the repo.
### Optional comments
_No response_
### Code of Conduct
- [x] I agree to follow the [Code of Conduct](https://github.com/JamieMason/syncpack/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start at syncpack's file-discovery entry point and the traversal used by `syncpack lint`, then reproduce the slowdown with a large gitignored directory or workspace globs such as `apps/*` and `libs/*`. Compare behavior with explicit `--source` flags and verify that lint output is unchanged while traversal avoids irrelevant directories and runtime improves.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, rust
- Domain
- cli, performance, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100