pingdotgg / pingdotgg/t3code

[Bug]: Model prints out internal dialog in answered message. Might just be hallucination.

Open
#9,642 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Before submitting
  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.
Area

Not sure

Steps to reproduce

This might just be a halucination of the actual model. But since I can't discern if it is the model or if the problem lies in the harness interacting with the model, I thought I should report it anyways. Unfortunately this is probably impossible to reproduce due to the random nature of LLMs.

Have a conversation with GPT-5.6-Sol on medium using the skill "grill-me". I used the Version 0.0.37 of t3.code hosted through npx.
Have a fairly full context window.
Use the grilling-skill from below
Have a conversation with it until it makes a mistake and starts outputting its internal dialog to the user.

grill me skill from @mattpocock.

---
name: grilling
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
---

Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it.

Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled: the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round.

Format a round like so:

Q1 - : <question body, might be multiple paragraphs, including multiple choices>

➡️


Q2 - : <question body, might be multiple paragraphs, including multiple choices>

➡️


Each round the user answers reshapes the tree: settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one.

Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it; don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report; ask the rest of the frontier now. The _decisions_ are the user's: put each to them and wait.

The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding.

Expected behavior

T3.codes interaction with the model and "internal" dialog should not be presented to the user.

Actual behavior

T3.code interaction with the model and "internal" dialog about a typo it made is presented to me and included in its answer.

First there is some swedish. Then there is a typo and an anxious rant about correcting it. This is how the complete answer ends.

. . . Mitt exempel: ”Okej, det blev så idag. Du var trött ochs inte ensam.” Hmm typo! Need final can't retract? We need correct before sending. Need phrase no false. "Okej, det blev så idag. Du var trött och försökte komma undan det jobbiga. Det var kanske inte bra för dig, men du är inte dålig för det. Vad kan vi göra nu?" That's a couple sentences but good. Need not overdo. Ensure only one question. Let's formulate. Sorry analysis crossed final? No, final channel currently content includes typo? I need edit mentally. final response clean.

The message ends with "final response clean."

Impact

Minor bug or occasional failure

Version or commit

No response

Environment

Node , Chrome Version 152.0.7977.65

Logs or stack traces

Screenshots, recordings, or supporting files

image.png

Workaround

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported conversation in t3.code 0.0.37 through npx with GPT-5.6-Sol, the grilling skill, and a full context window; the issue provides no source file or test entry point. Trace where the model response becomes the user-visible answer and verify that internal dialog, including the accidental final-response text, is not presented to the user.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.