anthropics / anthropics/claude-code

[Bug] Dual-use safeguard false positives on legitimate defensive security review; mid-session auto-model-switch breaks continuity

オープン
#87,802 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
area:model area:security duplicate platform:macos
主要言語
Python
スター
145k
フォーク
23.1k
PR マージ指標
PR 指標を取得中

説明

**Bug Description**

Subject: Fable 5 dual-use safeguard repeatedly flags a legitimate defensive security review; mid-session auto-switch to Opus 4.8 makes it unusable

Product: Claude Code (CLI) · Model: Claude Fable 5 (claude-fable-5) · Date: 2026-08-19

About me & our product:

I'm the Co-Founder and CEO of ImposterHunter AI LTD (https://imposterhunter.com/), where we are preparing to release our LLM Shield: a runtime AI security layer that detects adversarial attacks based on deception mechanism and intent (and not on known keywords or phrases), and that catches attacks across the full conversation, regardless of the language or encoding used. Every identified attack returns the Attack Category, the actual "poison" in the conversation, the risk score, a recommended action, and a full reasoning on why this is an attack. It actually aims to address the exact problem/bug we are reporting here with the current Claude "Guardrails," which render the Claude LLMs "unusable" for this workflow.

What I'm doing:

I'm running a defensive security review of my own product (an LLM defense solution / content-classification service) in my own AWS account and my own private Git repo. The work is:

1. Tenant-isolation review (IAM, network, provisioning/teardown, secret & key lifecycle) which is pure static code + infra reading;
2. Checking whether the API leaks proprietary classification metadata in response bodies or error paths;
3. Minting keys for my own tenant and running benign and auth-boundary calls against my own endpoint.

All of it is against infrastructure I own and operate. There is no third-party target.

The problem:

False positives on plainly legitimate security work: The dual-use safeguard has fired repeatedly this session (including on the assistant's own replies, where it was narrowing scope and describing static code review. This is exactly the "flags legitimate coding / cybersecurity tasks" case your own flag notice acknowledges.

The mid-session auto-switch is the wrong remedy.
Each flag silently swaps the model (Fable 5 to Opus 4.8) in the middle of a run.
It breaks continuity, drops the reasoning thread, and forces a manual `/model fable` to recover, every few turns. Even where a flag is defensible, yanking the model mid-session is disruptive out of proportion to the signal.

What would help, in priority order:

a. Lower the false-positive rate of the dual-use layer on clearly-scoped, own-asset defensive security review (pentest / guardrail red-teaming of a system the user owns).
b. Make model-switch-on-flag configurable: let a flag degrade to a warning rather than an automatic swap, or let me opt out at the session level, so one probabilistic flag doesn't derail a whole run.
c. Publish a supported path / guidance for security researchers reviewing their own systems with Claude Code, so this workflow is recognized rather than re-adjudicated every session.
d. Consider evaluating a solution like the one we're providing in scoring the nature of the conversation and which provides direct mitigation and even (potentially) a real explanations to users when they might attempt to use Claude with 'bad' intentions.

Impact:

As-is, Fable 5 is impractical for this workflow because of the repeated interruptions. I want to keep using Fable 5 rather than route around it. Please treat this as a false-positive report on the dual-use classifier plus a UX report on the auto-switch behavior.

Reproduction:
The session transcript sent with this feedback contains the flagged turns.

P.S. If interested, we would love to give your team an eval key and let you run your chosen attacks against our LLM Shield. Hopefully this would help the entire Claude community.

Thank you in advance, Dragos Balhui

Reproduction: the session transcript sent with this feedback contains the flagged turns.

**Environment Info**
- Platform: darwin
- Terminal: vscode
- Version: 2.1.232
- Feedback ID: a2fc2e56-b937-44fd-b030-4d5d10bdd189

**Errors**
```json
[]
```

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start with the session transcript referenced in the report and reproduce the flagged turns using Claude Code 2.1.232, noting when the session switches models. Check the /model fable recovery path and verify that legitimate own-asset defensive review no longer causes repeated disruptive switches, or that the requested behavior is clearly configurable.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, python
領域
cli, security
issue の種類
バグ
難易度
5/5
見積もり時間
1週間以上
活発さ
活発
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。