openai / openai/codex-security

Improve ChatGPT allowance visibility during long-running scans with stale estimated cost

Open
#29 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:auth area:cli area:cost documentation enhancement priority:p2
Dominant language
TypeScript
Stars
10.8k
Forks
801
Avg merge
1d 8h
Merged PRs (30d)
257

Description

Summary

The issue is the lack of actionable usage visibility while authenticating with ChatGPT. During a long-running request, the CLI's estimated cost remained unchanged for approximately 40 minutes while the ChatGPT weekly Codex allowance continued decreasing until it was exhausted.

The CLI did not communicate the relationship—or lack of relationship—between its USD-denominated estimate and the ChatGPT plan allowance.

Environment

  • Codex Security CLI: 0.1.1
  • Node.js: 22.23.1
  • npm: 10.9.8
  • OS: macOS
  • Authentication: stored ChatGPT credentials
  • codex-security login status reported Logged in using ChatGPT

The repository identity and contents are intentionally omitted.

Reproduction steps

Run a deep full-repository scan:

cd /path/to/private-repository

codex-security scan . \
  --mode deep \
  --model gpt-5.6-sol \
  --codex 'model_reasoning_effort="xhigh"' \
  --output-dir /path/to/private-output \
  --max-cost 10

The CLI initially reported regular estimated-cost updates:

[00:14] Estimated cost: $0.120945 of $10.00 limit
[00:52] Estimated cost: $0.293178 of $10.00 limit
[01:15] Estimated cost: $0.355227 of $10.00 limit
[01:55] Estimated cost: $0.55516 of $10.00 limit
[02:03] Estimated cost: $0.591838 of $10.00 limit

After that, the CLI remained at Running scan without further cost or phase updates.

I canceled the scan after approximately 42 minutes:

[42:47] Running scan
^C
codex-security: Scan canceled by Ctrl-C.
codex-security: Partial output was kept at /path/to/private-output.

Observed behavior

At approximately 12 minutes, the ChatGPT Codex usage interface showed about 37% of the weekly allowance used.

By the time I canceled the scan at approximately 42 minutes, the weekly allowance had been exhausted.

Throughout that period, the CLI continued displaying $0.591838 of $10.00 limit as its latest estimated cost.

The scan was intentionally configured with a high budget and an expensive model/reasoning combination. High token usage was therefore expected.

What was not clear was:

  • whether the in-flight request was still accumulating usage;
  • whether the displayed estimate included the active request;
  • how the USD-denominated estimate related to the ChatGPT weekly allowance;
  • how quickly the weekly allowance was being consumed;
  • whether stopping the scan was necessary to preserve the remaining allowance.

Expected behavior

When using stored ChatGPT credentials, the CLI should provide clearer usage observability.

Possible improvements include:

  1. Clearly state that --max-cost is an estimated USD-equivalent limit and does not directly cap the ChatGPT plan allowance.
  2. Indicate when the displayed estimate excludes an active in-flight request.
  3. Avoid presenting a stale estimate as if it were current.
  4. Display a status such as Model request in progress; usage estimate pending during long requests.
  5. Periodically report token, credit, or plan-allowance consumption when that information is available.
  6. Warn when the CLI cannot determine the remaining ChatGPT plan allowance.
  7. Optionally provide a guard based on tokens, Codex credits, or remaining ChatGPT allowance rather than only estimated USD cost.

The goal is not to prevent users from choosing expensive scans. It is to let them make an informed decision while the scan is running.

Impact

The scan exhausted the weekly Codex allowance without completing.

The main usability problem was not the amount consumed, but that the CLI provided no actionable indication that the remaining weekly allowance was close to exhaustion.

This is particularly confusing with ChatGPT authentication because:

  • the CLI displays an estimated cost in USD;
  • the user sees a weekly allowance in the ChatGPT interface;
  • no mapping between those two measurements is shown;
  • the CLI estimate may remain unchanged during a long-running request.

As a result, the user cannot reliably decide whether to continue or cancel a scan.

Additional observation

A later standard scan with a smaller budget stopped close to the configured limit:

codex-security scan . \
  --model gpt-5.6-terra \
  --codex 'model_reasoning_effort="medium"' \
  --output-dir /path/to/another-private-output \
  --max-cost 0.25

It stopped with:

Estimated cost: $0.25063 of $0.25 limit
codex-security: Scan stopped: estimated cost $0.25063 exceeded the $0.25 limit

This may indicate that the budget check and displayed estimate update correctly between completed requests, while usage from a long in-flight request is not visible until that request finishes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the scan command's --max-cost handling and the status output that reports estimated cost and scan phases. Reproduce a long-running request and determine when estimates stop updating and whether active requests are included. Done should make stale or unavailable usage explicit and clearly distinguish the USD estimate from ChatGPT plan allowance consumption.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
cli, observability
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.