`jp c grep` cannot match text split across adjacent chat response events
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 16
- Forks
- 3
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 121
Description
Search operates on one event at a time, but the terminal and web renderers join consecutive same-kind chat responses into a single region. Text that a user can plainly read in jp c print is therefore not always findable with jp c grep.
shared::search::event_lines (crates/jp_cli/src/shared/search.rs:92) takes a single &EventKind and splits that event's own text with .lines(). Its signature makes cross-event matching impossible. collect_scope_hits (crates/jp_cli/src/cmd/conversation/grep.rs:754) then runs the match and --context pipeline over those per-event lines.
The concrete case is Anthropic's redacted thinking. A thinking block is interrupted by an opaque redacted_thinking block and resumes afterwards, which lands on disk as three reasoning events splitting a word in half:
{"type": "chat_response", "reasoning": "...I can test this directly by ver"}
{"type": "chat_response", "reasoning": "", "metadata": {"anthropic_redacted_thinking": "..."}}
{"type": "chat_response", "reasoning": "ifying the return value."}
The three events cannot be merged upstream: each thinking block carries its own signature and a ConversationEvent holds one anthropic_thinking_signature, so merging drops a signature and the next request fails validation. The renderers join them at display time instead.
This is not specific to reasoning. Any two adjacent same-kind text responses have the same gap.
Expected Behavior
jp c grep verifying finds the match, because "verifying" is what the conversation shows the user.
Actual Behavior
No match. The stored text holds ver in one event and ifying in the next, and search never sees them adjacent.
Reproduce
Requires a conversation containing an Anthropic redacted-thinking split, or an equivalent hand-built stream of two adjacent reasoning events whose text breaks mid-word.
jp c printon the conversation — the word renders whole.jp c grep <word>— no hit.
Predates #919. The stored shape has always been three events; that PR changed only how they render, so grep behaves identically before and after it.
Proposed Solution
The fix needs a decision before code, because both options change user-visible contracts.
Option A — search regions. Coalesce adjacent same-kind chat responses before matching, mirroring what the renderers do. Matches then behave the way the conversation reads. Costs: a hit's coordinate currently locates an event, so region hits need a coordinate that means something (first event of the region? a range?), and --context lines would cross event boundaries, changing what --context 3 returns.
Option B — leave search event-scoped, document the limit. Zero risk, and the gap stays.
I lean A, on the grounds that a search that cannot find visible text is surprising in the way that matters (principle of least astonishment), and the coordinate question has a defensible answer (the region's first event). But the output shape of jp c grep and its --context semantics are things users script against, so the change wants deciding deliberately rather than in passing.
Worth noting what is not on the table: a shared "logical region" projection used by every consumer. serve-web deliberately works on raw serde_json::Value events without depending on jp_conversation types (crates/plugins/command/serve-web/src/render.rs:1-6), matching the direction of moving plugins out of core. A projection in jp_conversation could serve replay and grep but not serve-web.
Tasks
- Decide between region-scoped search (A) and documenting the limit (B)
- If A: define what coordinate a region hit reports, and whether
--contextmay cross event boundaries - Add a failing test with two adjacent reasoning events splitting a word mid-token
- Implement the chosen option
- Update
jp c grephelp text and any docs describing its scope model
Resources
https://github.com/dcdpr/jp/pull/919
https://github.com/dcdpr/jp/pull/919#discussion_r3690006034
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with shared::search::event_lines in crates/jp_cli/src/shared/search.rs and collect_scope_hits in crates/jp_cli/src/cmd/conversation/grep.rs, then compare their behavior with the renderer joining described in the issue. Decide the search-region and coordinate/context contract before adding a failing test for adjacent reasoning events that split a word. Done means the chosen behavior is implemented, the grep help or scope documentation is updated, and the test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100