bug: MemoryEntry display crashes with UnicodeEncodeError on Windows cp125x consoles
- Dominant language
- Python
- Stars
- 73
- Forks
- 89
- Avg merge
- 14d 58m
- Merged PRs (30d)
- 2
Description
## Summary
`MemoryEntry.__str__()` embeds literal Unicode box-drawing characters (U+2514 `└`, U+2500 `─`, U+251C `├`) in its output. When `display=True` (the default), `Console().print(panel)` writes these to `sys.stdout`. On Windows installations where `sys.stdout.encoding` is a cp125x variant, these codepoints have no encoding mapping and raise `UnicodeEncodeError`, crashing the simulation.
This affects all memory backends (`STMemory`, `STLTMemory`, `LTMemory`) since they all call `MemoryEntry.display()`.
## Reproduction
```python
# repro.py — no API keys needed, no LLM calls, pure rendering
from rich.console import Console
from rich.panel import Panel
# Simulate a cp1252 stdout (default on Windows en-US when piped/redirected)
import io, sys
buf = io.BytesIO()
sys.stdout = io.TextIOWrapper(buf, encoding="cp1252", errors="strict")
# Replicate MemoryEntry.__str__() output
content = "[bold cyan][Observation][/bold cyan]\n [blue]\u2514\u2500[/blue] [cyan]price :[/cyan] $20"
panel = Panel(content, title="Step 1 | Agent 0", title_align="left",
border_style="bright_blue", padding=(0, 1))
Console().print(panel)
```
```
stdout encoding: cp1252
+- Step 1 | Test 0 ------...---+ <- Rich degrades ITS borders to ASCII (safe_box)
| [Observation] |
| CRASHES: 'charmap' codec can't encode characters in position 0-1: character maps to
```
Rich's `safe_box` correctly degrades its own panel borders to ASCII. But the box-drawing characters in `__str__()` are **literal content text**, not Rich decoration -- Rich cannot degrade them, so they pass through to `sys.stdout.write()` and fail.
## Expected behavior
Memory display renders without crashing on all platforms and all Windows code pages.
## Actual behavior
`UnicodeEncodeError: 'charmap' codec can't encode character '\u2514'` -- simulation crashes.
## Who is affected
`sys.stdout.encoding` on Windows depends on the system locale. The box-drawing characters used (`└─├`) crash on all cp125x variants:
| Encoding | Region | `└─├` (U+2514/2500/251C) |
|---|---|---|
| cp1252 | Western Europe (en, fr, de, es, pt, it, nl, ...) | CRASH |
| cp1250 | Central Europe (cs, pl, hu, sk, ro, hr) | CRASH |
| cp1251 | Cyrillic (ru, bg, uk) | CRASH |
| cp1253 | Greek | CRASH |
| cp1254 | Turkish | CRASH |
| cp1255 | Hebrew | CRASH |
| cp1256 | Arabic | CRASH |
| cp874 | Thai | CRASH |
| cp437 / cp850 | DOS/legacy US-EU console | safe |
| cp932 / cp936 / cp949 | Japanese / Chinese / Korean | safe |
| cp65001 | UTF-8 (opt-in) | safe |
This triggers in common scenarios: piping output, running inside VS Code / PyCharm / Jupyter, CI runners -- anywhere Python falls back to `locale.getpreferredencoding()` which returns the cp125x variant.
## Options considered
### 1. Replace with `•` (U+2022 bullet) -- rejected
`•` is safe in cp125x but **crashes on cp437, cp850, cp932, cp936, cp949** (DOS-legacy and Asian Windows). This trades one crash for another.
| Encoding | `└─├` | `•` (U+2022) |
|---|---|---|
| cp1252 (Western Windows) | CRASH | safe |
| cp437 (DOS/legacy US) | safe | CRASH |
| cp932 (Japanese) | safe | CRASH |
| cp936 (Chinese) | safe | CRASH |
### 2. Reconfigure `sys.stdout` to UTF-8 -- rejected
Libraries should not mutate global `sys.stdout` encoding. Side effects on user code, not guaranteed to work (buffer may not support `reconfigure`), and does not fix the root cause.
### 3. Wrap `console.print()` in try/except -- rejected
Hides the root cause, swallows the error silently, adds complexity. The user sees no output instead of a crash -- still broken, just quieter.
### 4. Replace with ASCII tree markers (`+--`, `|--`) -- recommended
`+`, `-`, `|` are pure ASCII (bytes 0x2B, 0x2D, 0x7C). Safe in every encoding ever created -- cp125x, cp437, cp9xx, cp65001, all of them. Preserves the tree structure that `└─`/`├─` provided. This is the same rendering that `tree --charset=ASCII` and `npm list` use.
Before:
```
[Observation]
└─ price : $20
└─ seller : Bob
[Action]
├─ (1)
│ └─ offer : $25
├─ (2)
```
After:
```
[Observation]
+-- price : $20
+-- seller : Bob
[Action]
|-- (1)
| +-- offer : $25
|-- (2)
```
7 lines changed in `memory.py`, zero logic change, zero behavior change.
## Proposed fix
File: `mesa_llm/memory/memory.py`, `MemoryEntry.__str__()` (lines 52-90)
Replace `└─` with `+--` and `├─` with `|--`. Add a test verifying `str(entry)` output is encodable across all major Windows code pages.
Contributor guide
Research direction
Start in mesa_llm/memory/memory.py at MemoryEntry.__str__() (lines 52-90), then inspect how MemoryEntry.display() renders the result. Verify the replacement tree markers remain ASCII-encodable across the listed Windows code pages and add the requested str(entry) encoding test. Done means memory display renders without UnicodeEncodeError on those encodings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 85/100