technicalpickles / technicalpickles/cq

cq: surface thinking-block content from messages (currently dropped in array_msgs)

Open
#48 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
11
Forks
1
Avg merge
4h 43m
Merged PRs (30d)
22

Description

claude_messages_sql's array_msgs CTE in src/views.rs (~lines 123-126) picks exactly one content block per message to extract into messages.text:

(SELECT json_extract_string(item, '$.text')
 FROM (SELECT UNNEST(CAST(json_extract(json, '$.message.content') AS JSON[])) AS item)
 WHERE json_extract_string(item, '$.type') = 'text'
 LIMIT 1) AS text,

The WHERE item.type = 'text' filter means thinking blocks never survive into messages, or anything downstream of it (trace, search, etc.). This isn't the empty-signature "encrypted thinking" case from some fixtures. A live grep of real transcripts under ~/.claude/projects/*/*.jsonl shows thinking content that's often substantial and non-trivial, e.g. "I should look up Jira issue AIDEV-509 to find its associated epic...". There's real content being silently dropped.

Design questions this issue should resolve before implementation, not after:

  1. Storage shape: a new thinking_text column alongside messages.text, or a separate view/CTE? A single message can carry both a thinking block and a text block, so array_msgs can't just swap which type it filters for.
  2. Scope: does this belong on messages at all, or does tool_calls/the trace Span/Gap model in src/trace/mod.rs also want thinking content? If messages is the only consumer, this is a cq-core feature, not a trace one.
  3. If thinking content does need to reach the trace emitter eventually, the existing args.detail mechanism (from #46/#47, in src/trace/perfetto.rs) should cover it once the data exists in messages, no new mechanism needed.
  4. Size/redaction: thinking text can be long, and occasionally sensitive in the same way tool args can be. Not investigated yet.

Scoping only. No implementation in this issue.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/views.rs at the array_msgs CTE and trace its consumers, then inspect src/trace/mod.rs and src/trace/perfetto.rs for existing thinking-content paths. Compare the available storage options, trace scope, and size/redaction concerns using real transcript examples. Done means the storage and scope decisions are documented clearly enough to define a follow-up implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.