CopilotKit / CopilotKit/outpost

Decide the generator's empty-response behaviour

Open
#178 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: ai roadmap roadmap: next
Dominant language
TypeScript
Stars
7
Forks
3
Avg merge
7d 16h
Merged PRs (30d)
15

Description

packages/outpost/ai/src/generator.ts was changed on #170's branch to throw when the model returns no usable text:

const responseText = extractResponseText(message.content);
if (!responseText.trim()) {
    throw new Error('Model response contained no usable text');
}

Previously it returned an empty string. The change came in alongside a genuine fix — the generator only read message.content[0], so a multi-block response silently lost every block after the first — but the throw is a separate, unrequested behaviour change and deserves its own decision.

What it changes

Every caller of the generator is affected, not just the support-response path. An empty or tool-only model response now becomes a thrown error, which becomes a failed job, which after MAX_JOB_ATTEMPTS walks to DEAD_LETTER.

The case for it

Fail-loud is usually right. Publishing an empty response to a customer is worse than failing visibly, and an empty completion nearly always means something upstream is wrong — a bad prompt, a refusal, a truncated stream, a tool-only turn being treated as final.

The case against it

  • It converts a recoverable oddity into a terminal job failure. Retrying an empty completion is often the right move, and three retries then dead-letter is a heavy response.
  • A tool-only response is legitimately empty of text. If any pipeline path uses tool calls, this throws on a correct response.
  • The blast radius is every generator caller, decided in a PR about something else.

What to decide

  1. Is throwing correct, or should an empty response be retried, or returned and handled by the caller?
  2. If it throws, should it be a distinct error type so the queue can classify it as retryable rather than dead-lettering?
  3. Should the check be !responseText.trim(), or narrower — e.g. only when there are no content blocks at all, so a tool-only turn passes through?

Small change, real reach. Worth an explicit call rather than inheriting it from a multi-block bug fix.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/outpost/ai/src/generator.ts, then trace every caller of the generator and the job retry path described in the issue. Decide whether empty or tool-only responses should throw, retry, or return, and whether errors need queue classification. Done means the policy is explicit and the generator and affected callers consistently implement it.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.