openai / openai/codex-security

scans match --all builds unbounded quadratic history payloads

Open
#136 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:cli area:cost area:reports bug priority:p2
Dominant language
TypeScript
Stars
10.8k
Forks
801
Avg merge
1d 8h
Merged PRs (30d)
257

Description

Summary

Automatic scan-history matching sends all findings from every earlier unmatched scan in one model request for each later scan.

Relevant code:

For scan n, the batch includes findings from scans 1..n-1. Across a history, materialized and submitted data therefore grows approximately quadratically. There is no finding-count or prompt-byte limit.

The matching plan is first returned through runWorkbench, which has a 4 MiB stdout limit, so a large history can fail locally before comparison begins. Smaller payloads can still exceed model context or cause unexpectedly high matching cost.

Why this matters

The feature becomes less reliable and more expensive as the repository accumulates the history it is designed to compare. One finding-heavy scan can make every later batch substantially larger.

Expected behavior

Matching should operate within documented, deterministic input limits and remain usable for long-lived repositories.

Suggested direction

Process bounded finding batches, persist intermediate pair results, and reconcile cross-batch groups deterministically. If safe batching cannot preserve semantics, enforce an explicit history/finding ceiling with guidance to match smaller ranges.

Add scale tests that verify prompt/workbench payload limits and stable behavior across multiple batches.

Found by static audit of upstream main at 9c7634b.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace matching-plan materialization in sdk/typescript/_bundled_plugin/scripts/workbench_scan_history.py, then follow flattening in sdk/typescript/src/cli.ts and prompt serialization in sdk/typescript/src/scan-comparison.ts. Add scale tests covering workbench and prompt limits and stable behavior across multiple batches; done means long histories remain within documented deterministic limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
cli, performance, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.