mpfaffenberger / mpfaffenberger/code_puppy
Grouped small DRY extractions: surrogate-sanitizing file read copy-pasted 6x across tools/ and file_permission_handler; _history.py duplicates its tool-call-id set collection
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 814
- Forks
- 278
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 76
Description
Severity: Low-Medium (small, mechanical extractions; one already shows drift)
Two independent micro-duplications in scope, grouped because each is too small for its own issue:
1. "Read file + sanitize surrogates" block exists 6 times
The same open-read-sanitize sequence:
with open(file_path, "r", encoding="utf-8", errors="surrogateescape") as f:
original = f.read()
try:
original = original.encode("utf-8", errors="surrogatepass").decode("utf-8", errors="replace")
except (UnicodeEncodeError, UnicodeDecodeError):
pass
appears at:
tools/file_modifications.py:226-234(_delete_snippet_from_file)tools/file_modifications.py:277-286(_replace_in_file) — jscpd pair 221-234 ↔ 271-286tools/file_modifications.py:381-388(_write_to_file, overwrite branch)tools/file_operations.py:500-513(_read_file) — this copy has a better fallback (aggressive per-char surrogate strip) that the other five lackplugins/file_permission_handler/register_callbacks.py:82and:141
file_operations.py:559-585 already has _sanitize_string() doing exactly the sanitize half. Extract once, e.g. in tools/common.py next to atomic_write_text (its natural mirror):
def read_text_sanitized(file_path: str | Path) -> str:
"""Read UTF-8 text, converting lone surrogates to U+FFFD."""
…and have all six call sites use it, so the robust fallback applies everywhere instead of only in read_file. The sibling clone file_modifications.py:241-256 ↔ 335-350 (snippet-check → unified_diff → atomic_write_text → result-dict scaffold) is structurally covered by the compute_replacements/EditResult core proposed in #459 — fixing that issue with a shared EditResult.diff removes those pairs too; noting here only so the jscpd hits are accounted for.
2. agents/_history.py: tool-call/return id collection duplicated
prune_interrupted_tool_calls (297-306) and has_pending_tool_calls (333-342) contain the identical 10-line loop building tool_call_ids / tool_return_ids sets via _classify_tool_part. Extract:
def _collect_tool_ids(messages: List[ModelMessage]) -> tuple[Set[str], Set[str]]:
"""Return (call_ids, return_ids) across all message parts."""
Then prune_interrupted_tool_calls is mismatched = calls ^ returns and has_pending_tool_calls is bool(calls - returns) — the two functions' different set algebra becomes the visible part, instead of being buried below identical loops. (The cross-file fork of _history walkers into the context_indicator plugin is already tracked in #457; this is purely the within-file pair.)
Filed by Zen Reviewer A (code-puppy-60635a) — DRY review round
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tools/common.py and compare the six read-and-sanitize call sites in tools/file_modifications.py, tools/file_operations.py, and plugins/file_permission_handler/register_callbacks.py. Then inspect agents/_history.py at prune_interrupted_tool_calls and has_pending_tool_calls. Done means both groups use shared helpers while preserving the robust sanitization fallback and each function's distinct set logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 64/100