mpfaffenberger / mpfaffenberger/code_puppy
file_operations._list_files: O(n^2) duplicate-dir scan, 4-5 syscalls per entry, double sort; ignore pattern '**/.*' contradicts its own comment
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 814
- Forks
- 278
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 76
Description
File: code_puppy/tools/file_operations.py - _list_files (~lines 150-300)
Severity: Medium (performance + correctness)
- O(n^2) duplicate-directory check. For every file returned by ripgrep, the code synthesizes parent directory entries and checks for duplicates with a linear scan over the entire results list, inside a loop over path components:
for i in range(len(path_parts)):
partial_path = os.sep.join(path_parts[: i + 1])
if not any(f.path == partial_path and f.type == "directory" for f in results):
results.append(...)
On a repo with tens of thousands of files this is quadratic (each any() walks all accumulated entries). Fix: keep a seen_dirs: set[str] alongside results - O(1) membership.
-
Redundant stat syscalls per entry:
os.path.exists,os.path.isfile/isdir,os.path.getsize, thenos.statagain - 4-5 syscalls per file where oneos.stat(with try/except) provides everything. -
Results are sorted twice with the same
_sort_key(once to buildfile_entriesfor the UI message, once to build the text output, ~lines 415 and 440). Sort once into a local. -
should_ignore_pathignores ALL hidden files by design confusion (code_puppy/tools/common.py:437): the pattern list contains"**/.*"with the comment "# Commented out as it might be too aggressive" - but it is NOT commented out. Either remove the pattern or fix the comment; right now the comment actively misleads. -
_list_filesand_grepduplicate the rg-discovery block (shutil.which + venv probing) and the tempfile-ignore-file dance verbatim - extract a_find_ripgrep()and_write_ignore_file()helper (also reduces the two separate temp-file cleanup paths).
Filed by Zen Reviewer B (code-puppy-60635a)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in code_puppy/tools/file_operations.py at _list_files and _grep, then inspect should_ignore_path in code_puppy/tools/common.py. Trace the ripgrep discovery, temporary ignore-file handling, directory-entry construction, metadata lookup, and output sorting. Done means the listed performance issues are addressed, shared discovery and cleanup logic is extracted, and the '**/.*' comment and behavior agree.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100