block / block/buzz

Windows agent sessions can silently replace Unicode punctuation with ? before signing messages

Open
#6,527 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

# Windows agent sessions can silently replace Unicode punctuation with `?` before signing Buzz messages

## Summary

On Windows, an agent session that sends generated message content from Windows PowerShell 5.1 through `buzz messages send --content -` without explicitly setting UTF-8 pipe encoding can silently replace non-ASCII characters with ASCII question marks before the Nostr event is built and signed.

This is easy to miss because the command succeeds, the relay accepts the event, and Buzz Desktop faithfully renders the already-corrupted event. The corruption is permanent in message history and can vary between per-channel sessions of the same agent.

## Observed behavior

Intended text:

```text
The standalone laptop is a solid call — it solves the availability concern.
No desktop environment needed — less to patch, less to break.
```

Stored/rendered text:

```text
The standalone laptop is a solid call ? it solves the availability concern.
No desktop environment needed ? less to patch, less to break.
```

The substitution repeats at every intended em dash. This is not mojibake such as `—`, and it is not a Desktop rendering error: the outbound PowerShell/native-process boundary has already converted `U+2014` to `U+003F` before submission.

## Environment

- Windows 11
- Windows PowerShell `5.1.26100.9168`
- Buzz managed ACP agent sessions
- Buzz CLI invoked by the agent harness
- One independent agent session per channel

## Reproduction

From Windows PowerShell 5.1, pipe Unicode prose to the CLI without first setting `$OutputEncoding` to UTF-8:

```powershell
$message = 'alpha — beta “quoted” café'
$message | buzz messages send --channel --content -
```

Fetch the resulting signed event and inspect its `content` code points. Depending on the active legacy encoding, unsupported characters can be stored as `?` (`U+003F`) or otherwise corrupted.

The immediate workaround is:

```powershell
$OutputEncoding = [Console]::OutputEncoding = [Text.UTF8Encoding]::new()
$message | buzz messages send --channel --content -
```

However, a prompt/workspace instruction is insufficient because each channel has an independent, potentially long-running agent session. One session can comply while another session of the same agent continues using the unsafe encoding.

## Expected behavior

Every Windows agent session should preserve generated Unicode text byte-for-byte through CLI submission without relying on prompt compliance or per-command shell setup.

## Proposed fix

Initialize every Windows PowerShell process spawned by `buzz-acp` with explicit BOM-less UTF-8 input/output encoding before agent commands execute:

```powershell
$OutputEncoding = [Console]::OutputEncoding = [Text.UTF8Encoding]::new()
```

Also consider removing the shell encoding boundary from human-facing message submission:

- provide a `buzz messages send --content-file ` option that reads UTF-8 explicitly; or
- have the harness invoke the CLI directly and write UTF-8 bytes to stdin rather than constructing PowerShell command arguments.

## Regression test

Add a Windows end-to-end test that sends the following through the same harness path used by managed agents:

```text
em dash — curly quotes “test” accented café CJK 測試 emoji 🐝
```

Fetch the accepted event and assert exact Unicode scalar values in `content`, not only successful command exit or visual rendering. Run the test for two independent channel sessions to cover per-session initialization.

## Impact

- Signed message history can be permanently altered before relay ingestion.
- The failure is silent: the CLI reports success.
- Multiple sessions of one agent can behave differently.
- Human readers may incorrectly attribute the problem to Desktop rendering or relay storage.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the PowerShell 5.1 pipeline with `buzz messages send --content -`, then trace the Windows process setup in `buzz-acp` and the managed-agent harness. Add the Windows end-to-end regression test described in the issue, using two independent channel sessions, and verify the fetched signed event contains the exact Unicode scalar values for the supplied punctuation, accents, CJK text, and emoji.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell, rust
Domain
cli, operating-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.