anthropics / anthropics/claude-agent-sdk-python

max_buffer_size and import byte limits count characters instead of UTF-8 bytes

Aberta
#1,165 1 comentário 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
8.1k
Forks
1.3k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

### Summary

Several limits documented and reported as byte limits use `len(str)`, which counts Unicode code points rather than encoded UTF-8 bytes.

### Affected code

- `src/claude_agent_sdk/_internal/transport/subprocess_cli.py:68-90` — `_LineFramer`
- `src/claude_agent_sdk/_internal/transport/subprocess_cli.py:800-808` — stderr buffering
- `src/claude_agent_sdk/_internal/transport/subprocess_cli.py:970-987` — stdout message guard
- `src/claude_agent_sdk/_internal/session_import.py:128-142` — import batching

### Current behavior

The framer increments `pending_len` with `len(chunk)`, and completed lines are checked with `len(line)`. Error messages describe the result as bytes. Session import similarly says it flushes at 1 MiB of line bytes but increments with `len(line)`.

For multibyte UTF-8 text, the actual byte count can be significantly larger. A string of emoji, for example, occupies about four UTF-8 bytes per Python character.

### Why this matters

- `max_buffer_size` does not enforce the promised wire/memory boundary.
- A nominal 1 MiB configuration can accept roughly 4 MiB of UTF-8 content before rejection.
- stderr partial-line buffers and SessionStore append batches can grow well beyond their intended byte thresholds.
- Operators relying on these settings for memory protection receive misleading error measurements.

### Expected behavior

A setting and error described as bytes should count encoded bytes consistently.

### Possible fix

The most robust approach is to frame raw bytes and decode complete lines afterward. Alternatively, maintain an exact incremental UTF-8 byte count alongside text chunks and use `len(line.encode("utf-8"))` for completed lines.

Tests should use multibyte input immediately below and above the configured byte boundary, including chunk splits across Unicode content.

### Environment

- Repository revision: current `main` audit at SDK version 0.2.128
- Bundled CLI version: 2.1.220
- Python test suite: 1,291 passed, 5 skipped
- Ruff and mypy: clean

I searched the existing issues and pull requests using the affected symbols and behavior before filing this.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.