[Bug]: Model prints out internal dialog in answered message. Might just be hallucination.
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 23k
- Forks
- 5.9k
- Avg merge
- 11h 14m
- Merged PRs (30d)
- 357
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
Not sure
Steps to reproduce
This might just be a halucination of the actual model. But since I can't discern if it is the model or if the problem lies in the harness interacting with the model, I thought I should report it anyways. Unfortunately this is probably impossible to reproduce due to the random nature of LLMs.
Have a conversation with GPT-5.6-Sol on medium using the skill "grill-me". I used the Version 0.0.37 of t3.code hosted through npx.
Have a fairly full context window.
Use the grilling-skill from below
Have a conversation with it until it makes a mistake and starts outputting its internal dialog to the user.
grill me skill from @mattpocock.
--- name: grilling description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases. --- Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it. Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled: the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round. Format a round like so:❓ Q1 - : <question body, might be multiple paragraphs, including multiple choices>
➡️
❓ Q2 - : <question body, might be multiple paragraphs, including multiple choices>
➡️
Each round the user answers reshapes the tree: settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one. Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it; don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report; ask the rest of the frontier now. The _decisions_ are the user's: put each to them and wait. The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding.
Expected behavior
T3.codes interaction with the model and "internal" dialog should not be presented to the user.
Actual behavior
T3.code interaction with the model and "internal" dialog about a typo it made is presented to me and included in its answer.
First there is some swedish. Then there is a typo and an anxious rant about correcting it. This is how the complete answer ends.
. . . Mitt exempel: ”Okej, det blev så idag. Du var trött ochs inte ensam.” Hmm typo! Need final can't retract? We need correct before sending. Need phrase no false. "Okej, det blev så idag. Du var trött och försökte komma undan det jobbiga. Det var kanske inte bra för dig, men du är inte dålig för det. Vad kan vi göra nu?" That's a couple sentences but good. Need not overdo. Ensure only one question. Let's formulate. Sorry analysis crossed final? No, final channel currently content includes typo? I need edit mentally. final response clean.
The message ends with "final response clean."
Impact
Minor bug or occasional failure
Version or commit
No response
Environment
Node , Chrome Version 152.0.7977.65
Logs or stack traces
Screenshots, recordings, or supporting files
Workaround
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported conversation in t3.code 0.0.37 through npx with GPT-5.6-Sol, the grilling skill, and a full context window; the issue provides no source file or test entry point. Trace where the model response becomes the user-visible answer and verify that internal dialog, including the accidental final-response text, is not presented to the user.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100