[Image generation] Harmless prompt repeatedly false-positive blocked as erotic content
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Summary
Image generation repeatedly rejects a completely harmless prompt as potentially violating nudity/sexual/erotic-content guardrails.
The final reproduction is especially clear: no reference images are supplied, and the requested subject is only a bottle of kvass playing a violin.
Environment
- ChatGPT Plus
- ChatGPT web
- Windows 10 / Firefox
- Date reproduced: 2026-08-26
Reproduction
-
In a conversation, attempt an image-generation request that receives a sexual-content safety rejection.
-
Remove all reference images from the subsequent image-generation call.
-
Reduce the request to a harmless prompt such as:
A realistic bottle of kvass playing a realistic violin.The Russian user wording was simply:
бутылка кваса играет на скрипке -
Invoke image generation again.
-
Generation is rejected with:
We’re so sorry, but the prompt may violate our guardrails around nudity, sexuality, or erotic content. If you think we got it wrong, please retry or edit your prompt.
-
Retrying with another harmless formulation, including a request to use an ordinary kvass-bottle image and add arms + a violin, produces the same rejection.
Expected behavior
Generate an ordinary humorous image of an anthropomorphic beverage bottle playing a violin.
Actual behavior
The image generator repeatedly classifies the request as possible erotic content despite the final request containing only:
- a beverage bottle
- arms
- a violin
- no nudity
- no sexual activity
- no erotic description
- no reference images in the final generation attempt
Why this appears to be a bug
A realistic bottle of kvass playing a realistic violin. has no reasonable sexual interpretation.
The important part is that the false positive persists after reference images are removed and the generation request is replaced by a plainly harmless prompt. This suggests that an earlier safety signal or unrelated conversational context may be contaminating subsequent image-safety classification, or that a prior classification is being retained/cached across generation attempts.
That makes recovery from a false positive effectively impossible inside the same conversation.
Suggested investigation
Please check whether image-generation safety classification can incorrectly inherit/carry over moderation state from earlier image-generation attempts or unrelated conversation context.
A replacement text-only prompt with no references should be independently re-evaluated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file, test, or entry point is identified. Start by reproducing the ChatGPT web sequence: trigger a sexual-content rejection, remove references, then submit the kvass-and-violin prompt; done means the harmless prompt is independently evaluated and generates successfully without inherited moderation state.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, security
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100