rtk grep: invalid UTF-8 in a searched file corrupts file-count/attribution for other matches in the same file (exit 0, no warning)

Summary

rtk grep (and therefore the Claude Code PreToolUse auto-rewrite of a plain
grep -n PATTERN FILE command) can silently return structurally wrong
results — wrong "N matches in N files" header, matches attributed to no
filename at all, and match lines missing a leading chunk of their real
content — when the searched file contains any invalid/incomplete
multi-byte UTF-8 byte sequence on a line that precedes another matching
line in the same file. The command still exits 0, so nothing signals that
anything went wrong.

This reproduces 100% of the time on Linux/aarch64 with rtk 0.38.0 (the
installed version I tested against). I could not run the current
v0.49.0 release binary on this host (rtk-aarch64-unknown-linux-gnu.tar.gz
requires GLIBC_2.39+, this host has glibc 2.35), but I read the v0.49.0
source (src/cmds/system/search.rs, src/core/stream.rs,
src/core/utils.rs) and the same parse_match_line / format_match_line /
by_file grouping pipeline that plausibly causes this is still present
there, unchanged in shape. Please treat the version as unconfirmed for
v0.49.0 and confirm on your end.

Why this matters for the Claude Code hook use case

rtk hook claude / rtk hook check silently rewrites a plain
grep -n PATTERN FILE Bash tool call into rtk grep -n PATTERN FILE (see
repro below). The agent never sees the original grep invocation or its
real output — only rtk's (sometimes wrong) reformatted version, with exit
code 0 and no warning. An agent (or a human) reading this output has no
way to know the file/line/content attribution may be wrong.

Minimal, deterministic repro

$ xxd repro.txt
00000000: 4d41 5443 4831 2076 616c 6964 2061 7363  MATCH1 valid asc
00000010: 6969 206c 696e 650a 4d41 5443 4832 2074  ii line.MATCH2 t
00000020: 7275 6e63 6174 6564 2075 7466 383a 20e2  runcated utf8: .
00000030: 8220 7461 696c 2074 6578 740a 4d41 5443  . tail text.MATC
00000040: 4833 2076 616c 6964 2065 6d6f 6a69 3a20  H3 valid emoji: 
00000050: f09f 8e89 2074 6169 6c20 7465 7874 0a    .... tail text.

i.e. three lines:

MATCH1 valid ascii line
MATCH2 truncated utf8: <0xE2 0x82> tail text     <- 2-byte incomplete UTF-8 sequence (missing 3rd continuation byte)
MATCH3 valid emoji: 🎉 tail text

Confirm the Claude Code hook rewrite fires on this exact command:

$ rtk hook check grep -n MATCH repro.txt
rtk grep -n MATCH repro.txt

$ echo '{"tool_name":"Bash","tool_input":{"command":"grep -n MATCH repro.txt"}}' | rtk hook claude
{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecisionReason":"RTK auto-rewrite","updatedInput":{"command":"rtk grep -n MATCH repro.txt"}}}

Actual (rtk grep -n MATCH repro.txt, run repeatedly — fully
deterministic):

2 matches in 2 files:

3:0:🎉 tail text
repro.txt:1:MATCH1 valid ascii line

Expected (approximated by real grep -a -n, which shows what a
faithful "compact but correct" rewrite should preserve — one match per
matching line, correct file, correct line number, correct content):

1:MATCH1 valid ascii line
2:MATCH2 truncated utf8: � tail text
3:MATCH3 valid emoji: 🎉 tail text

Observed defects in the actual output, all at once:

1. Wrong file count: "2 matches in 2 files" — there is exactly one
   file (repro.txt), passed as a single positional path (no -r, no
   directory).
2. Missing filename attribution: the first displayed line is 3:0:🎉 tail text instead of repro.txt:3:MATCH3 valid emoji: 🎉 tail text — no
   filename shown, and a spurious :0: appears where a path: prefix
   should be.
3. Truncated/misaligned content: the same line loses MATCH3 valid emoji: from the front of the match content, showing only the tail
   🎉 tail text.
4. A match silently vanishes: MATCH2's line (the one containing the
   invalid UTF-8 bytes) is dropped entirely — not shown even with a
   replacement character, not counted, no indication anything was skipped.
5. Wrong ordering: the (corrupted) line for MATCH3 (physically line 3
   in the file) is printed before the line for MATCH1 (physically line
   1).

I confirmed this is not a general "line 1 vs line 3 alphanumeric string
sort" artifact — with a by_file grouping of size 1 (as it should be here),
the printed order should follow file-then-line order deterministically for
that one file's entries; getting entries 1 and 3 split into apparently two
different groups, printed out of line order, only makes sense if the
extracted "file" key differs between the two surviving matches (e.g. one
match parses to file="repro.txt", the other to file="" or some other
wrong value, giving two different HashMap<String, Vec<..>> buckets that
then sort ahead of/behind each other alphabetically instead of being one
bucket printed in file order).

What does NOT reproduce it (ruled out / bisected)

• A file with one single lone continuation byte (e.g. \x80 on its own,
  surrounded by ASCII) on a line by itself: that one line's match is
  silently dropped (same defect #4 above, minus #1-#3 — filename/count stay
  correct). So some corruption already happens with the mildest possible
  invalid byte, but the full file-count/attribution corruption needs a
  slightly "worse" invalid sequence (a genuinely incomplete/truncated
  multi-byte lead byte, e.g. \xE2\x82, \xF0\x9F\x8E, \xC0\x80 all
  reproduce the full corruption; see below).
• Grepping a directory containing one dirty file among several clean
  ones: the dirty file's bad line is dropped, but the other files and
  their content are unaffected, and the total "N files" count is correct.
  The corruption in this report is specific to a match that survives
  within the same file as an earlier invalid-UTF-8 line.
• rtk find on a directory containing a file with invalid UTF-8 bytes in
  its filename: correctly lists it (lossily) and the file/dir counts stay
  correct — this bug looks specific to rtk grep's content/match parsing,
  not rtk find.
• I also independently hit the same symptom signature (missing filename,
  spurious N:0: prefix, truncated leading content) on a completely clean,
  100%-valid-UTF-8 real Rust source file (81KB, no invalid bytes at all)
  using a plain literal single-term pattern with exactly one match — so
  invalid UTF-8 content is at least one trigger, but very possibly not
  the only one; the shared symptom strongly suggests one shared root cause
  in the match-line parsing/grouping path rather than two unrelated bugs.
  I was not able to reduce that second case to a small attached repro in
  the time I had, but can try to help narrow it further if useful.

Source-level pointers (read against tag v0.49.0, latest stable)

rtk grep shells out to the real grep/rg binary and re-parses its
output — it does not reimplement search natively
(src/cmds/system/search.rs:265-295):

fn parse_flags(self) -> &'static [&'static str] {
    match self {
        Engine::Grep => &["-n", "-H", "-I", "--null"],
        Engine::Rg => &["-n", "--with-filename", "--null"],
    }
}

Running that exact grep invocation by hand on repro.txt produces
perfectly well-formed, NUL-delimited output (verified with xxd):

$ grep -n -H -I --null -e MATCH -- repro.txt | xxd
00000000: 7265 7072 6f2e 7478 7400 313a 4d41 5443  repro.txt.1:MATC
...                                                  (MATCH1 line)
00000020: 696e 650a 7265 7072 6f2e 7478 7400 333a  ine.repro.txt.3:
...                                                  (MATCH3 line, no MATCH2 line — -I already suppressed it)

(Interesting side finding: -I, which rtk unconditionally injects, makes
grep treat this file as "binary" and skip the invalid-UTF-8 line even
though it contains no literal NUL byte — that's a real GNU grep behavior
difference from plain grep -n the agent actually typed, worth being aware
of separately, but not this report's main defect.)

Since the real subprocess output here is already clean, the corruption must
be introduced on rtk's own side, in one (or a combination) of:

• decode_process_output / decode_mixed / decode_line in
  src/core/utils.rs:675-734 — the byte→String decode of the captured
  child stdout, which takes a line-by-line lossy-UTF-8 fallback path
  whenever the whole captured buffer fails std::str::from_utf8;
• parse_match_line (src/cmds/system/search.rs:737-749), a regex
  ^([^\x00]+)\x00(\d+)([:-])(.*)$ that depends on an intact literal NUL
  byte surviving decode to split filename from line content;
• the by_file: HashMap<String, Vec<(usize,bool,String)>> grouping loop
  and show_file = by_file.len() > 1 || show_file(...) computation
  (src/cmds/system/search.rs:595-670), which is what actually produces
  the "N matches in N files" header and the file-count-driven decision of
  whether to print a path: prefix at all.

I read through these but couldn't pin the exact faulty line without a debug
build/instrumented binary — happy to help narrow further if that's useful,
but wanted to file the reproducible symptom + implicated subsystem now
rather than sit on it.

Possibly related existing issues

• #2360 ("rtk hook claude: Bash tool output corrupted on macOS
  (cat/sed/grep return mangled lines)") and #2176 ("rtk hook claude
  (v0.40.0, Windows) intermittently returns STDOUT not faithful to the real
  command output") describe the same family of symptom (grep -n output
  with dropped/renumbered/mislabeled lines, worked around with
  rtk proxy/rtk run) but both are platform-scoped, intermittent, and
  explicitly "couldn't isolate a minimal repro." This report is
  Linux/aarch64, 100% deterministic, and platform-agnostic in its
  mechanism (pure string/byte handling, not OS-specific IO timing), so it
  may be a distinct root cause that happens to produce a similar-looking
  symptom, or it may be one concrete trigger behind those broader reports.
  I'm not certain which, so linking rather than asserting either way.

Environment

• rtk 0.38.0 (~/.local/bin/rtk), Linux aarch64 (Orange Pi / RK3588,
  Ubuntu 22.04, glibc 2.35)
• /usr/bin/grep GNU grep 3.7 (no rg installed — confirmed rtk grep
  uses the grep engine on this host)
• Reproduced directly via rtk grep and via the actual Claude Code
  PreToolUse hook path (rtk hook claude / rtk hook check)
• LANG=LC_ALL=en_US.UTF-8 in the shell/agent environment (ruled out
  locale-dependent grep -I binary-detection as the explanation — see
  "what does NOT reproduce it" above)

Suggested severity

This is silent data corruption in a tool whose entire purpose is to feed
compacted-but-faithful command output to an LLM agent that treats it as
ground truth for further decisions (edits, diagnosis, "no more matches
found" conclusions, etc.), with exit 0 and no error indication. A single
file anywhere in a grep -rn-searched tree that happens to contain
non-UTF-8 bytes (common: logs, minified/generated assets, copy-pasted
smart-quote text, stray binary content) can corrupt result attribution for
other matches found in that same file. Workaround for anyone hitting this
today is rtk proxy <cmd> (bypasses rtk's rewrite/parsing entirely), same
as the workaround noted in #2360/#2176.

Reproduce the issue with the provided repro.txt and inspect src/core/utils.rs:675-734, src/cmds/system/search.rs:265-295 and :595-670, and parse_match_line at :737-749. Trace how NUL-delimited grep output is decoded and grouped, then verify that invalid UTF-8 does not alter filenames, line numbers, content, ordering, or counts and that the command reports results faithfully.