openai / openai/codex

[Image generation] Harmless prompt repeatedly false-positive blocked as erotic content

Open
#40,816 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug imagen safety-check
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

Image generation repeatedly rejects a completely harmless prompt as potentially violating nudity/sexual/erotic-content guardrails.

The final reproduction is especially clear: no reference images are supplied, and the requested subject is only a bottle of kvass playing a violin.

Environment

  • ChatGPT Plus
  • ChatGPT web
  • Windows 10 / Firefox
  • Date reproduced: 2026-08-26

Reproduction

  1. In a conversation, attempt an image-generation request that receives a sexual-content safety rejection.

  2. Remove all reference images from the subsequent image-generation call.

  3. Reduce the request to a harmless prompt such as:

    A realistic bottle of kvass playing a realistic violin.

    The Russian user wording was simply:

    бутылка кваса играет на скрипке

  4. Invoke image generation again.

  5. Generation is rejected with:

    We’re so sorry, but the prompt may violate our guardrails around nudity, sexuality, or erotic content. If you think we got it wrong, please retry or edit your prompt.

  6. Retrying with another harmless formulation, including a request to use an ordinary kvass-bottle image and add arms + a violin, produces the same rejection.

Expected behavior

Generate an ordinary humorous image of an anthropomorphic beverage bottle playing a violin.

Actual behavior

The image generator repeatedly classifies the request as possible erotic content despite the final request containing only:

  • a beverage bottle
  • arms
  • a violin
  • no nudity
  • no sexual activity
  • no erotic description
  • no reference images in the final generation attempt

Why this appears to be a bug

A realistic bottle of kvass playing a realistic violin. has no reasonable sexual interpretation.

The important part is that the false positive persists after reference images are removed and the generation request is replaced by a plainly harmless prompt. This suggests that an earlier safety signal or unrelated conversational context may be contaminating subsequent image-safety classification, or that a prior classification is being retained/cached across generation attempts.

That makes recovery from a false positive effectively impossible inside the same conversation.

Suggested investigation

Please check whether image-generation safety classification can incorrectly inherit/carry over moderation state from earlier image-generation attempts or unrelated conversation context.

A replacement text-only prompt with no references should be independently re-evaluated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file, test, or entry point is identified. Start by reproducing the ChatGPT web sequence: trigger a sexual-content rejection, remove references, then submit the kvass-and-violin prompt; done means the harmless prompt is independently evaluated and generates successfully without inherited moderation state.

Written by the indexing model from the issue text.

Assessment

Domain
ai, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.