adafruit / adafruit/cookiecutter-adafruit-circuitpython
pre-commit pylint checks are doing too much work
- Dominant language
- Python
- Stars
- 27
- Forks
- 39
- PR merge metrics
- No merged PRs in 30d
Description
I noticed that in adafruit_datetime, the pre-commit check could take a long time, especially for the "tests" step. Furthermore, all 4 of my CPU cores were active.
I believe this is because by default, pre-commit
* assumes that the program accepts filenames on its commandline/argv
* runs multiple processes in parallel if there are multiple files
Since pylint needs to get a view of all the source files it's checking in order to do proper code duplication checks, we make our own list of files to pylint with `find` and ignore the positional arguments that are given. But unless we also specify `pass_filenames: false` pre-commit doesn't know about it and starts invoking the "pylint all files" command once for each file!
This change is one I'm testing locally in adafruit_datetime:
```
From 10ebce0339182073dc0aabd010f73df00ae348a6 Mon Sep 17 00:00:00 2001
From: Jeff Epler
Date: Mon, 22 Mar 2021 20:34:25 -0500
Subject: [PATCH] pre-commit: don't do too much work
---
.pre-commit-config.yaml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index cce4c7b..96f0626 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -32,6 +32,7 @@ repos:
entry: /usr/bin/env bash -c
args: ['([[ ! -d "examples" ]] || for example in $(find . -path "./examples/*.py"); do pylint --disable=missing-docstring,invalid-name $example; done)']
language: system
+ pass_filenames: false
- repo: local
hooks:
- id: pylint_tests
@@ -40,3 +41,4 @@ repos:
entry: /usr/bin/env bash -c
args: ['([[ ! -d "tests" ]] || for test in $(find . -path "./tests/*.py"); do pylint --disable=missing-docstring $test; done)']
language: system
+ pass_filenames: false
--
2.29.2
```
If we want to make a change like this we'll have to apply it with adabot to existing repos as well.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in .pre-commit-config.yaml, especially the local pylint hooks for examples and tests and their find-based entries. Run pre-commit on a generated library to confirm each pylint hook runs once rather than once per file; done when the template change can be propagated to existing repositories with adabot.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, tooling
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100