anthropics / anthropics/skills

docx/pptx/xlsx: redlining.py decodes git diff output with the locale codec — silent mojibake or None stdout on Windows

Offen
#1,707 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
176k
Forks
20.8k
Ø Merge
7 Std. 21 Min.
Gemergte PRs (30 T.)
5

Beschreibung

## Summary

`scripts/office/validators/redlining.py` writes its two temp files as UTF-8, then decodes `git diff`'s output with the **locale** codec. On Windows (cp1252) that either corrupts the redline silently or kills the run with an exception that points nowhere near the cause.

The file is byte-identical in **docx**, **pptx** and **xlsx**, so all three skills are affected.

## Affected code

`skills/{docx,pptx,xlsx}/scripts/office/validators/redlining.py`

- L189-190 — both temp files written with an explicit `encoding="utf-8"`
- L192-204 — `subprocess.run([... "git", "diff", "-U0", "--no-index" ...], capture_output=True, text=True)`
- L221-232 — second call, same pattern

`text=True` decodes with `locale.getpreferredencoding(False)`, which is **cp1252** on a default Windows install — not the UTF-8 the files were just written in.

## Environment

Windows 11 (26200), CPython 3.14.4 (`win-amd64`), `locale.getpreferredencoding(False)` = `cp1252`, UTF-8 mode off, git 2.55.0.

## Reproduction

Replicating L189-204 exactly — write two UTF-8 temp files, diff them, collect added lines. Output below is `ascii()`-escaped so the console codec cannot confound the result:

```
devanagari
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u0928\u092e\u0938\u094d\u0924\u0947 \u0938\u0902\u0938\u093e\u0930']

arrow U+2192
text=True -> ['+after \xe2\u2020\u2019 y']
encoding=utf-8,errors=replace -> ['+after \u2192 y']

cyrillic U+050F
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u050f two']
```

Two distinct failure modes:

1. **Silent corruption.** `→` (U+2192) round-trips as `â†'` — no exception, and the redline is simply wrong.
2. **Crash with a misleading traceback.** Any character whose UTF-8 bytes land on an undefined cp1252 slot (0x81/0x8D/0x8F/0x90/0x9D) raises `UnicodeDecodeError` **on subprocess's reader thread**. `subprocess.run` does not re-raise it — `result.stdout` comes back as `None`, so the caller dies at the next line with:

```
AttributeError: 'NoneType' object has no attribute 'splitlines'
```

which gives no hint that an encoding problem occurred, or where.

Devanagari, Cyrillic, CJK and most non-Latin scripts hit mode 2. Curly quotes, em dashes and arrows — very common in the documents these skills exist to redline — hit mode 1.

## Suggested fix

Replace `text=True` with an explicit codec at both call sites, matching the encoding the files were written with two lines above:

```python
capture_output=True,
encoding="utf-8",
errors="replace",
```

Verified: all three cases above then round-trip correctly. `errors="replace"` also guarantees the reader thread cannot raise, so the `stdout is None` trap disappears.

Related, same root class but different mechanism (file I/O and `print` rather than subprocess decoding): #712, #1271, #1686.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.