anthropics / anthropics/skills

docx/pptx/xlsx: redlining.py decodes git diff output with the locale codec — silent mojibake or None stdout on Windows

Ouverte Adaptée aux débutants
#1,707 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
176k
Forks
20.9k
Merge moyen
7 h 21 min
PR mergées (30 j)
5

Description

## Summary

`scripts/office/validators/redlining.py` writes its two temp files as UTF-8, then decodes `git diff`'s output with the **locale** codec. On Windows (cp1252) that either corrupts the redline silently or kills the run with an exception that points nowhere near the cause.

The file is byte-identical in **docx**, **pptx** and **xlsx**, so all three skills are affected.

## Affected code

`skills/{docx,pptx,xlsx}/scripts/office/validators/redlining.py`

- L189-190 — both temp files written with an explicit `encoding="utf-8"`
- L192-204 — `subprocess.run([... "git", "diff", "-U0", "--no-index" ...], capture_output=True, text=True)`
- L221-232 — second call, same pattern

`text=True` decodes with `locale.getpreferredencoding(False)`, which is **cp1252** on a default Windows install — not the UTF-8 the files were just written in.

## Environment

Windows 11 (26200), CPython 3.14.4 (`win-amd64`), `locale.getpreferredencoding(False)` = `cp1252`, UTF-8 mode off, git 2.55.0.

## Reproduction

Replicating L189-204 exactly — write two UTF-8 temp files, diff them, collect added lines. Output below is `ascii()`-escaped so the console codec cannot confound the result:

```
devanagari
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u0928\u092e\u0938\u094d\u0924\u0947 \u0938\u0902\u0938\u093e\u0930']

arrow U+2192
text=True -> ['+after \xe2\u2020\u2019 y']
encoding=utf-8,errors=replace -> ['+after \u2192 y']

cyrillic U+050F
text=True -> STDOUT IS None -- decode raised on subprocess reader thread
encoding=utf-8,errors=replace -> ['+\u050f two']
```

Two distinct failure modes:

1. **Silent corruption.** `→` (U+2192) round-trips as `â†'` — no exception, and the redline is simply wrong.
2. **Crash with a misleading traceback.** Any character whose UTF-8 bytes land on an undefined cp1252 slot (0x81/0x8D/0x8F/0x90/0x9D) raises `UnicodeDecodeError` **on subprocess's reader thread**. `subprocess.run` does not re-raise it — `result.stdout` comes back as `None`, so the caller dies at the next line with:

```
AttributeError: 'NoneType' object has no attribute 'splitlines'
```

which gives no hint that an encoding problem occurred, or where.

Devanagari, Cyrillic, CJK and most non-Latin scripts hit mode 2. Curly quotes, em dashes and arrows — very common in the documents these skills exist to redline — hit mode 1.

## Suggested fix

Replace `text=True` with an explicit codec at both call sites, matching the encoding the files were written with two lines above:

```python
capture_output=True,
encoding="utf-8",
errors="replace",
```

Verified: all three cases above then round-trip correctly. `errors="replace"` also guarantees the reader thread cannot raise, so the `stdout is None` trap disappears.

Related, same root class but different mechanism (file I/O and `print` rather than subprocess decoding): #712, #1271, #1686.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Inspect the matching call sites in skills/docx/scripts/office/validators/redlining.py, skills/pptx/scripts/office/validators/redlining.py, and skills/xlsx/scripts/office/validators/redlining.py, especially L192-204 and L221-232. Reproduce the UTF-8 diff cases from the issue, then verify that all three validators preserve non-ASCII output and no longer produce None stdout or mojibake.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
git, python
Domaine
tooling
Type d'issue
Bug
Difficulté
2/5
Temps estimé
1-3 heures
Activité
Active
Clarté
Clairement spécifiée
Accessibilité débutants
88/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.