better way of processing the Content Warning
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
When a Content Warning is triggered by user input, the warning message is still being sent to the language model and a response is generated, instead of the conversation stopping.
**Steps to Reproduce**
1. Go to chat.lmsys.org
2. Enter a message that should trigger a Content Warning (e.g. explicit language, violence, etc.)
3. The Content Warning is displayed, and sent to the language model
4. A response to the Content Warning is returned
**Expected Behavior**
When a Content Warning is triggered, the conversation should stop and no further processing or response generation should occur from the language model.
**Actual Behavior**
The Content Warning is displayed, but the content warning message is still sent to the language model, which produces a response. This wastes resources and may provide no additional insight.
**Severity**
Medium - This is a functional bug that goes against expected behavior for Content Warnings. However, it does not expose any sensitive data.
**Suggested Fix**
Implement logic to stop processing the user's message as soon as a Content Warning is triggered. The conversation flow should be halted at that point, without sending the message to the language model.
**Additional Notes**
Continuing to process messages that trigger Content Warnings is inefficient and provides no value to the end user. Addressing this bug would improve the user experience and conserve resources.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named; start by tracing the chat flow that handles Content Warnings and the subsequent language-model request. Reproduce the reported case, then verify that a triggered warning stops the conversation before any response is generated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100