garrytan / garrytan/gstack

security: WARN verdict takes no defensive action

Open
#1,156 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

## Summary

When the ML prompt injection ensemble returns a `WARN` verdict (confidence >= 0.60 but below BLOCK threshold), the system logs the event but takes **no defensive action**. The agent continues executing with full tool access as if the scan returned `safe`.

## Current behavior

In `preSpawnSecurityCheck` (`sidebar-agent.ts`):

```typescript
const result = combineVerdict(signals);
if (result.verdict !== 'block') return false; // WARN falls through here
```

WARN is treated identically to safe — no user confirmation, no tool restriction, no rate limiting. The only trace is a log line and a shield icon update.

## Risk

A prompt injection payload that scores just below the BLOCK threshold (e.g., 0.70 confidence from one classifier, not cross-confirmed by the second) gets logged but otherwise runs unrestricted. The ML stack correctly identified it as suspicious, but the system discards that signal.

## Possible mitigations (not prescriptive)

- **User confirmation gate**: show the WARN in the sidepanel, require explicit "proceed anyway" before spawning
- **Tool restriction**: spawn the agent with a reduced allowlist (e.g., Read-only, no Bash/Write) on WARN
- **Rate limiting**: throttle WARN sessions to prevent automated probing
- **Elevated logging**: capture full payload hash + domain for WARN events (currently only BLOCK gets `logAttempt`)

Each has different UX tradeoffs — filing as issue for design discussion rather than prescribing a fix.

## Severity

HIGH — the ML stack has a detection tier (WARN) that produces no defensive output.

Contributor guide

Open the contributing guide

Research direction

Start in sidebar-agent.ts at preSpawnSecurityCheck and trace combineVerdict through the existing BLOCK handling. Review how WARN currently falls through, then resolve which defensive behavior is selected; done means WARN no longer proceeds with unrestricted tool access and the chosen behavior is covered by the project’s relevant checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.