anthropics / anthropics/skills
docx/pptx/xlsx: redlining.py decodes git diff output with the locale codec — silent mojibake or None stdout on Windows
- Langage dominant
- Python
- Étoiles
- 176k
- Forks
- 20.9k
- Merge moyen
- 7 h 21 min
- PR mergées (30 j)
- 5
Description
## Summary
`scripts/office/validators/redlining.py` writes its two temp files as UTF-8, then decodes `git diff`'s output with the **locale** codec. On Windows (cp1252) that either corrupts the redline silently or kills the run with an exception that points nowhere near the cause.
The file is byte-identical in **docx**, **pptx** and **xlsx**, so all three skills are affected.
## Affected code
`skills/{docx,pptx,xlsx}/scripts/office/validators/redlining.py`
- L189-190 — both temp files written with an explicit `encoding="utf-8"`
- L192-204 — `subprocess.run([... "git", "diff", "-U0", "--no-index" ...], capture_output=True, text=True)`
- L221-232 — second call, same pattern
`text=True` decodes with `locale.getpreferredencoding(False)`, which is **cp1252** on a default Windows install — not the UTF-8 the files were just written in.
## Environment
Windows 11 (26200), CPython 3.14.4 (`win-amd64`), `locale.getpreferredencoding(False)` = `cp1252`, UTF-8 mode off, git 2.55.0.
## Reproduction
Replicating L189-204 exactly — write two UTF-8 temp files, diff them, collect added lines. Output below is `ascii()`-escaped so the console codec cannot confound the result:
```
devanagari
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u0928\u092e\u0938\u094d\u0924\u0947 \u0938\u0902\u0938\u093e\u0930']
arrow U+2192
text=True -> ['+after \xe2\u2020\u2019 y']
encoding=utf-8,errors=replace -> ['+after \u2192 y']
cyrillic U+050F
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u050f two']
```
Two distinct failure modes:
1. **Silent corruption.** `→` (U+2192) round-trips as `â†'` — no exception, and the redline is simply wrong.
2. **Crash with a misleading traceback.** Any character whose UTF-8 bytes land on an undefined cp1252 slot (0x81/0x8D/0x8F/0x90/0x9D) raises `UnicodeDecodeError` **on subprocess's reader thread**. `subprocess.run` does not re-raise it — `result.stdout` comes back as `None`, so the caller dies at the next line with:
```
AttributeError: 'NoneType' object has no attribute 'splitlines'
```
which gives no hint that an encoding problem occurred, or where.
Devanagari, Cyrillic, CJK and most non-Latin scripts hit mode 2. Curly quotes, em dashes and arrows — very common in the documents these skills exist to redline — hit mode 1.
## Suggested fix
Replace `text=True` with an explicit codec at both call sites, matching the encoding the files were written with two lines above:
```python
capture_output=True,
encoding="utf-8",
errors="replace",
```
Verified: all three cases above then round-trip correctly. `errors="replace"` also guarantees the reader thread cannot raise, so the `stdout is None` trap disappears.
Related, same root class but different mechanism (file I/O and `print` rather than subprocess decoding): #712, #1271, #1686.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
Inspect the matching call sites in skills/docx/scripts/office/validators/redlining.py, skills/pptx/scripts/office/validators/redlining.py, and skills/xlsx/scripts/office/validators/redlining.py, especially L192-204 and L221-232. Reproduce the UTF-8 diff cases from the issue, then verify that all three validators preserve non-ASCII output and no longer produce None stdout or mojibake.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- git, python
- Domaine
- tooling
- Type d'issue
- Bug
- Difficulté
- 2/5
- Temps estimé
- 1-3 heures
- Activité
- Active
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 88/100