anthropics / anthropics/skills
docx/pptx/xlsx: redlining.py decodes git diff output with the locale codec — silent mojibake or None stdout on Windows
- Vorherrschende Sprache
- Python
- Sterne
- 176k
- Forks
- 20.8k
- Ø Merge
- 7 Std. 21 Min.
- Gemergte PRs (30 T.)
- 5
Beschreibung
## Summary
`scripts/office/validators/redlining.py` writes its two temp files as UTF-8, then decodes `git diff`'s output with the **locale** codec. On Windows (cp1252) that either corrupts the redline silently or kills the run with an exception that points nowhere near the cause.
The file is byte-identical in **docx**, **pptx** and **xlsx**, so all three skills are affected.
## Affected code
`skills/{docx,pptx,xlsx}/scripts/office/validators/redlining.py`
- L189-190 — both temp files written with an explicit `encoding="utf-8"`
- L192-204 — `subprocess.run([... "git", "diff", "-U0", "--no-index" ...], capture_output=True, text=True)`
- L221-232 — second call, same pattern
`text=True` decodes with `locale.getpreferredencoding(False)`, which is **cp1252** on a default Windows install — not the UTF-8 the files were just written in.
## Environment
Windows 11 (26200), CPython 3.14.4 (`win-amd64`), `locale.getpreferredencoding(False)` = `cp1252`, UTF-8 mode off, git 2.55.0.
## Reproduction
Replicating L189-204 exactly — write two UTF-8 temp files, diff them, collect added lines. Output below is `ascii()`-escaped so the console codec cannot confound the result:
```
devanagari
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u0928\u092e\u0938\u094d\u0924\u0947 \u0938\u0902\u0938\u093e\u0930']
arrow U+2192
text=True -> ['+after \xe2\u2020\u2019 y']
encoding=utf-8,errors=replace -> ['+after \u2192 y']
cyrillic U+050F
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u050f two']
```
Two distinct failure modes:
1. **Silent corruption.** `→` (U+2192) round-trips as `â†'` — no exception, and the redline is simply wrong.
2. **Crash with a misleading traceback.** Any character whose UTF-8 bytes land on an undefined cp1252 slot (0x81/0x8D/0x8F/0x90/0x9D) raises `UnicodeDecodeError` **on subprocess's reader thread**. `subprocess.run` does not re-raise it — `result.stdout` comes back as `None`, so the caller dies at the next line with:
```
AttributeError: 'NoneType' object has no attribute 'splitlines'
```
which gives no hint that an encoding problem occurred, or where.
Devanagari, Cyrillic, CJK and most non-Latin scripts hit mode 2. Curly quotes, em dashes and arrows — very common in the documents these skills exist to redline — hit mode 1.
## Suggested fix
Replace `text=True` with an explicit codec at both call sites, matching the encoding the files were written with two lines above:
```python
capture_output=True,
encoding="utf-8",
errors="replace",
```
Verified: all three cases above then round-trip correctly. `errors="replace"` also guarantees the reader thread cannot raise, so the `stdout is None` trap disappears.
Related, same root class but different mechanism (file I/O and `print` rather than subprocess decoding): #712, #1271, #1686.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.