Better external dependency identification (ignore dotfiles directories by default)
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 424
- PR merge metrics
- No merged PRs in 30d
Description
I was hitting major issues because of the .venv and .mypy directories in my project.
- The common issue with virtual environments is [already covered in this issue](https://github.com/bndr/pipreqs/issues/214), but ignoring dot-prefixed directories would fix this for 80% of people.
- Mypy stores cache files in directories named by the package being analyzed, so pipreqs thinks they are local packages and does not register them as external dependencies.
I can't think of a reason why source files should be in a dot directory, but if that's not a surefire rule, an alternative could be to copy [the default ignored paths that `isort` uses](https://pycqa.github.io/isort/docs/configuration/options.html#skip) ([source](https://github.com/PyCQA/isort/blob/main/isort/settings.py#L72-L94)):
```
DEFAULT_SKIP: FrozenSet[str] = frozenset(
{
".venv",
"venv",
".tox",
".eggs",
".git",
".hg",
".mypy_cache",
".nox",
".svn",
".bzr",
"_build",
"buck-out",
"build",
"dist",
".pants.d",
".direnv",
"node_modules",
"__pypackages__",
".pytype",
}
)
```
I'd suggest taking a page from their book on [recognizing modules types](https://github.com/PyCQA/isort/blob/main/isort/place.py#L15) as well. Just pass in the module name, and ignore anything that's not "THIRDPARTY". It's not 100%, but it's tried and mostly true.
Contributor guide
Research direction
The issue does not name pipreqs source files or tests. Start by locating the dependency traversal and module-identification entry points, then compare the proposed behavior with isort's settings.py skip list and place.py module classification; done means dot-prefixed directories no longer create false local dependencies while external dependencies remain identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100