OpenHands / OpenHands/software-agent-sdk

[Bug]: Concurrent stdout and stderr chunks can reuse BashOutput order values

Open
#4,659 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

agent-execution architecture bug priority:medium ready-for-dev
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Bug Description

The two stream-reader coroutines in openhands-agent-server/openhands/agent_server/bash_service.py:288-346 share output_order. Each builds and persists a BashOutput using the current value, awaits _pub_sub(output_event), and increments only afterward at :315-325.

Because stdout and stderr readers run concurrently, the peer can execute during that await and allocate the same order. This is reachable with WebSocket subscribers because publication awaits websocket.send_json.

Duplicate persisted/public order values make reconstruction ambiguous and can cause clients polling with order__gt to skip output.

Expected Behavior

Every output event for a command should have a unique, contiguous order, with the final exit event last.

Actual Behavior

An independent, self-contained reproducer is posted in the issue comments. Save
it as /tmp/repro_4659.py, then run:

uv run python /tmp/repro_4659.py

On SDK commit 6d3881035, it exercises real BashEventService, subprocesses writing more than 1 MiB to both streams, persistence, and a yielding Subscriber; no mocks and prints:

output_orders=[0, 0, 2]
duplicate_orders=[0]
stdout_chars=1048577
stderr_chars=1048577
persisted_event_files=4
Suggested Implementation

Give each command one small synchronous sequencer/emitter. Reserve and increment the next order before the first await, then persist and publish. Keep stdout/stderr buffers local to their readers and send the final event through the same sequencer.

Acceptance Criteria
  • Interleaved oversized stdout and stderr produce unique contiguous order values.
  • A deliberately yielding subscriber cannot cause duplicate orders.
  • Both streams reconstruct exactly and the final exit event is last.
  • Persistence-before-publication, chunk size, timeout behavior, and REST/event schemas remain unchanged.
Version

Current main at 6d3881035. Found by static concurrency analysis during a read-only audit.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read openhands-agent-server/openhands/agent_server/bash_service.py:288-346, then save the comment reproducer as /tmp/repro_4659.py and run it with uv run python. Trace how stdout, stderr, and the final exit event are persisted and published. Done means oversized interleaved streams produce unique contiguous orders, reconstruction is exact, the exit event is last, and the listed schemas and behavior remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.