pytest-dev / pytest-dev/pluggy
Tracing crashes with UnicodeEncodeError on surrogate escapes in hook arguments/results
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 160
- Avg merge
- 21h 4m
- Merged PRs (30d)
- 6
Description
Problem
When pluggy tracing is enabled (e.g. pytest's --debug flag), hook arguments or return values containing surrogate escape characters (like \ud800) cause a UnicodeEncodeError crash. This is because _format_message in _tracing.py uses str() to format values, and the resulting string with literal surrogates cannot be written to most output targets.
Reported originally as pytest-dev/pytest#13750.
Analysis
_format_message currently formats two kinds of data identically using str():
- Structural labels passed as positional args — e.g.
"finish","-->", hook names — which are always safe ASCII strings - Python values — hook kwargs values (rendered via the
extradict) and hook return values — which may contain arbitrary data including surrogates
The fix needs to apply repr() only to the value positions, not to structural labels.
Intended Solution
-
In
_tracing.pyline 38 — use{value!r}for the extra/kwargs dict values:lines.append(f"{indent} {name}: {value!r}\n")This makes kwargs values in trace output show their type (e.g.
'lfplugin'instead oflfplugin,PosixPath('/foo')instead of/foo) and safely escapes surrogates. -
In
_manager.pyline 506 — userepr()on the hook result before passing it as a trace arg:hooktrace("finish", hook_name, "-->", repr(outcome.get_result()))This ensures the result value is safely formatted, while
"finish",hook_name, and"-->"remain plainstr()-formatted (via_format_message's existingmap(str, args)). -
Do NOT change the
content = " ".join(map(str, args))line in_format_message. Keepingstr()there preserves readable structural output without quoting labels. The caller (_manager.py) is responsible for pre-formatting any unsafe values withrepr().
This avoids the double-repr problem and keeps trace output readable:
finish pytest_runtest_call --> '\ud800' [hook]
config: <Config object at 0x...>
plugin_name: 'lfplugin'
Rather than the over-quoted version that blanket repr() in _format_message would produce:
'finish' 'pytest_runtest_call' '-->' "'\\ ud800'" [hook]
References
- pytest-dev/pytest#13750
- #627 (PR with discussion leading to this distilled approach)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with _format_message in _tracing.py around line 38 and the hook result tracing call in _manager.py around line 506. Verify tracing with surrogate-containing hook arguments and results no longer raises UnicodeEncodeError, while structural labels remain readable and values are safely represented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 85/100