CopilotKit / CopilotKit/outpost

AI response handler: the SHADOW_MODE compare still fails open

Open
#157 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: ai roadmap roadmap: now
Dominant language
TypeScript
Stars
7
Forks
3
Avg merge
7d 16h
Merged PRs (30d)
15

Description

Three pre-existing issues in packages/outpost/queue/src/handlers/ai-response.ts, all verified during the CR of #150 and all outside that PR's subject.

1. escalated: true is reported even when the escalation never enqueued

The escalation createJob is wrapped in a try/catch that logs and continues, but the handler's result still reports escalated: true. So a ticket whose escalation failed to enqueue looks escalated in the job record while no human was ever routed to it — the same "the record disagrees with what happened" class as #148.

2. A null description is stringified as the literal "null"

The classifier prompt interpolates ticket.description directly. Ticket.description is non-null in the schema today, but the AI pipeline path also handles tickets built from partial rows, and the guard costs nothing.

3. SHADOW_MODE === 'true' fails open

The strict comparison means SHADOW_MODE=TRUE, =1, or =yes all read as "not shadow mode" and post to real community surfaces. Shadow mode is a safety flag; it should fail closed on any truthy-looking value it does not recognize, and preferably log the value it rejected. This pattern is repo-wide, so fixing it centrally (a shared isShadowMode() helper) is probably better than patching one call site.

Found by: CR rounds 1-2 on #150 (slots 1-2, 2-1).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/outpost/queue/src/handlers/ai-response.ts and inspect the escalation createJob catch, description interpolation, and SHADOW_MODE check. Trace the repo-wide shadow-mode pattern before deciding whether a shared isShadowMode() helper fits. Done means failed enqueues are not reported as escalated, null descriptions are not stringified as "null", and unrecognized truthy values do not post to real community surfaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.