anthropics / anthropics/claude-code-security-review

False-positive filter calls retired model claude-3-5-haiku-20241022 (404) — filter silently degraded; suggest configurable filter-model input

Aperta
#114 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
6.2k
Fork
683
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Summary

The action's false-positive filter validates against a hardcoded Claude model identifier — **`claude-3-5-haiku-20241022`** — that has been retired by Anthropic. The validation 404s on every run, the filter is effectively disabled (or its behavior on validation failure is at minimum unclear from logs), and the action proceeds without informing the consumer that filtering is non-functional. The workflow exits with status 0 ("success") regardless.

## Environment

- **Action SHA pinned:** `0c6a49f1fa56a1d472575da86a94dbc1edb78eda` (latest `main` at time of report, 2026-06-20).
- **Workflow `claude-model` input:** `claude-opus-4-8` (current Opus per Anthropic docs).
- **Trigger:** `pull_request: [opened, synchronize, reopened]`.
- **Private repo.** GitHub-hosted `ubuntu-latest` runner.

## Steps to reproduce

1. Install the action per the README's Quick Start.
2. Provide a valid, Claude-Code-enabled API key as `claude-api-key`.
3. Open any PR.
4. Inspect the uploaded `security-review-results` artifact.

## Observed

`claudecode-error.log` (full contents):
```
[REPO#PR] [claudecode.claude_api_client] Claude API client initialized successfully
[REPO#PR] [claudecode.claude_api_client] Claude API validation failed: Error code: 404 - {'type': 'error', 'error': {'type': 'not_found_error', 'message': 'model: claude-3-5-haiku-20241022'}, 'request_id': 'req_011CcEywN359csdBV8tnXjhD'}
[REPO#PR] [claudecode.findings_filter] Claude API validation failed: API validation failed: Error code: 404 - {'type': 'error', 'error': {'type': 'not_found_error', 'message': 'model: claude-3-5-haiku-20241022'}, 'request_id': 'req_011CcEywN359csdBV8tnXjhD'}
```

`claudecode-results.json` reports `review_completed: true` and `findings: []`, and the workflow exits 0.

The hardcoded identifier `claude-3-5-haiku-20241022` no longer resolves in the Anthropic API. Claude 3.5 Haiku has been superseded; the current Haiku identifier per [Anthropic's docs](https://docs.anthropic.com/en/docs/about-claude/models) is `claude-haiku-4-5-20251001` (Haiku 4.5).

## Expected

Either:
1. **Filter model is configurable** — add a `filter-model` input that defaults to a current Anthropic Haiku model (e.g. `claude-haiku-4-5-20251001`) and is overridable by the consumer alongside the existing `claude-model` input.
2. **Filter model auto-tracks Anthropic's current Haiku** — the README's `claude-model` field already points readers at [models/overview#model-names](https://docs.anthropic.com/en/docs/about-claude/models/overview#model-names); the filter could resolve a sliding alias on init.
3. **Validation failure is fatal** — if the filter can't validate, fail loudly rather than silently proceed with the filter disabled (or proceed with raw findings unfiltered, but log a HIGH-visibility warning so consumers know the safety net is degraded).

## Impact

The action's value proposition includes false-positive filtering ("Advanced filtering removes low-impact or false positive prone findings to reduce noise" — README §How It Works). With the filter validation failing on every run, that step is effectively a no-op or unpredictable, and consumers reading a green check on the PR could be misled into thinking the full analysis pipeline ran cleanly.

For our usage we're treating the action as a backstop to a manual `/security-review` skill we run locally, so the impact for us is reduced confidence in the green check, not a primary security gap. Other consumers relying on the action as their primary security check may be more affected.

## Suggested fix sketch

```yaml
# action.yml
inputs:
filter-model:
description: 'Claude model for false-positive filtering. Defaults to current Haiku.'
required: false
default: 'claude-haiku-4-5-20251001'
```

…with the corresponding env var passed through to `claudecode/findings_filter.py` / `claude_api_client.py`.

## Logs / artifacts

Happy to share the full artifact if useful — the relevant files are `claudecode-results.json`, `findings.json`, and `claudecode-error.log` from the `security-review-results` artifact of any post-deprecation run.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.