Responses API: completed responses are never removed from the global `responses` dict

What happens

responses_wrappers.py keeps a module-global dict:

responses: dict[str, TracedData] = {}   # line 199

Entries go in at three places (632 sync, 800 async, 1067 streaming). They come out at two (832 and 864), and both of those live inside the cancel wrappers. So any response that finishes normally stays in the dict for the life of the process.

We measured about 17 KiB retained per turn in a chat workload, roughly 166 MiB per 10k turns. Our pods were restarting on OOM. The dict survives gc.collect(), since the module global is a live reference.

Same code in 0.62.1, in 0.62.3, and on main today. I diffed all three, responses_wrappers.py is byte identical.

Why the entries are dead

The sync path writes unconditionally, then emits the span only for completed:

        responses[parsed_response.id] = traced_data     # 632
    except Exception:
        return response

    if parsed_response.status == "completed":           # 636
        ...
        span.end()                                      # 647, entry still in the dict

After span.end() nothing reads that entry again. The merge reads at 579 and 746 exist so a later retrieve on a still running response can recover start_time, input, tools and trace_context from the original create, which is the background polling flow. Once the span is out, there is nothing left to merge into.

Streaming is the clearest case. Line 1067 writes to the dict and the next three lines finish the span:

                    responses[parsed_response.id] = self._traced_data   # 1067

                set_data_attributes(self._traced_data, self._span)
                self._span.set_status(StatusCode.OK)
                self._span.end()

Nothing ever reads that entry. It is garbage from the moment it is written.

Two things that might look like they need the retention, and do not:

• previous_response_id never appears in the file, so multi turn chaining never looks anything up. Turn N+1 gets a fresh id and misses the dict.
• background never appears either. There is no special handling, though the background flow (create returns queued, retrieve polls until completed) is exactly what the merge is for.

A second bug in the same block

While tracing the reads I hit this: nothing guards against emitting a span twice. create, retrieve and parse all bind to responses_get_or_create_wrapper (v1/__init__.py:305-318), the gate at 636 and 804 only checks status == "completed", and TracedData carries no "already emitted" field. Calling retrieve on a response that already completed emits another span for the same call.

This matters for the fix. Delete the entry at span.end() and nothing else, and that duplicate span gets worse instead of disappearing: start_time falls back to the retrieve's own timestamp instead of the create time, input/instructions/tools come back empty because a bare retrieve carries none, and trace_context becomes the current context instead of the originating one.

Proposed fix

Two changes that belong together:

1. Delete the entry where the span is emitted (after 647 and 813), and drop the dead write at 1067 entirely.
2. Track emitted response ids in a bounded structure (LRU or TTL) and skip the emission block at 636/804 when the id is already there.

Change 1 alone leaves the degraded duplicate described above, which is why I would not land it by itself. Change 2 removes duplicate spans, which is a behavior change and the part worth your call before I open a PR.

The guard holds bare id strings, not TracedData with its input, output_blocks and tools, so bounding it costs little. Dropping an id from the guard just means a duplicate span again, which is today's behavior. Dropping an entry from the main dict would lose start_time and trace_context and degrade a real span, so the dict itself should stay exact rather than capacity bound.

I considered swapping the plain dict for a TTL or LRU cache, one line at 199. It caps growth instead of removing it, every completed response still sits there until eviction, the size is a guess about someone else's workload, evicting too early breaks an in flight background poll, and the duplicate spans stay. Happy to go that way if you prefer the smaller diff.

Tests

A regression test needs no cassette and no network. The leak shows up in module state, and the file already has unit tests in that style, for example test_responses_trace_context_propagation_unit (tests/traces/test_responses.py:562), which imports TracedData from the wrapper directly and asserts on a local InMemorySpanExporter. Driving the completed path N times and asserting len(responses) == 0 fits the same mold.

Worth noting for anyone reproducing: one cassette per test function, and no test sets allow_playback_repeats, so the VCR route would need a hand edited cassette. The unit style avoids that.

Unrelated, noticed in passing

The exception path at 514 and 685 reads non_sentinel_kwargs.get("response_id"), but response_id is positional in the SDK (openai/resources/responses/responses.py:1315, before the *). For the normal client.responses.retrieve("resp_abc") that lookup always misses. Separate from this issue, say the word and I will open another.

---

Glad to send the PR if the two part fix works for you.

Start with integrations/openai/lib/...? responses_wrappers.py and the create, retrieve, parse, and streaming paths described in the issue, then read tests/traces/test_responses.py near test_responses_trace_context_propagation_unit. Add regression coverage for completed responses and repeated retrievals; done means completed entries do not remain in responses and duplicate spans are prevented without breaking background polling.