mpfaffenberger / mpfaffenberger/code_puppy

Grouped small DRY extractions: surrogate-sanitizing file read copy-pasted 6x across tools/ and file_permission_handler; _history.py duplicates its tool-call-id set collection

Open
#463 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
814
Forks
278
Avg merge
2d 5h
Merged PRs (30d)
76

Description

Severity: Low-Medium (small, mechanical extractions; one already shows drift)

Two independent micro-duplications in scope, grouped because each is too small for its own issue:

1. "Read file + sanitize surrogates" block exists 6 times

The same open-read-sanitize sequence:

with open(file_path, "r", encoding="utf-8", errors="surrogateescape") as f:
    original = f.read()
try:
    original = original.encode("utf-8", errors="surrogatepass").decode("utf-8", errors="replace")
except (UnicodeEncodeError, UnicodeDecodeError):
    pass

appears at:

  • tools/file_modifications.py:226-234 (_delete_snippet_from_file)
  • tools/file_modifications.py:277-286 (_replace_in_file) — jscpd pair 221-234 ↔ 271-286
  • tools/file_modifications.py:381-388 (_write_to_file, overwrite branch)
  • tools/file_operations.py:500-513 (_read_file) — this copy has a better fallback (aggressive per-char surrogate strip) that the other five lack
  • plugins/file_permission_handler/register_callbacks.py:82 and :141

file_operations.py:559-585 already has _sanitize_string() doing exactly the sanitize half. Extract once, e.g. in tools/common.py next to atomic_write_text (its natural mirror):

def read_text_sanitized(file_path: str | Path) -> str:
    """Read UTF-8 text, converting lone surrogates to U+FFFD."""

…and have all six call sites use it, so the robust fallback applies everywhere instead of only in read_file. The sibling clone file_modifications.py:241-256 ↔ 335-350 (snippet-check → unified_diffatomic_write_text → result-dict scaffold) is structurally covered by the compute_replacements/EditResult core proposed in #459 — fixing that issue with a shared EditResult.diff removes those pairs too; noting here only so the jscpd hits are accounted for.

2. agents/_history.py: tool-call/return id collection duplicated

prune_interrupted_tool_calls (297-306) and has_pending_tool_calls (333-342) contain the identical 10-line loop building tool_call_ids / tool_return_ids sets via _classify_tool_part. Extract:

def _collect_tool_ids(messages: List[ModelMessage]) -> tuple[Set[str], Set[str]]:
    """Return (call_ids, return_ids) across all message parts."""

Then prune_interrupted_tool_calls is mismatched = calls ^ returns and has_pending_tool_calls is bool(calls - returns) — the two functions' different set algebra becomes the visible part, instead of being buried below identical loops. (The cross-file fork of _history walkers into the context_indicator plugin is already tracked in #457; this is purely the within-file pair.)

Filed by Zen Reviewer A (code-puppy-60635a) — DRY review round

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/common.py and compare the six read-and-sanitize call sites in tools/file_modifications.py, tools/file_operations.py, and plugins/file_permission_handler/register_callbacks.py. Then inspect agents/_history.py at prune_interrupted_tool_calls and has_pending_tool_calls. Done means both groups use shared helpers while preserving the robust sanitization fallback and each function's distinct set logic.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.