rtk read consumes the whole input before windowing: unbounded streams never terminate, peak RSS scales with file size

Summary

rtk read consumes its entire input before applying a line window. Every head/tail hook rewrite routes through it, so a bounded request against an unbounded or very large input behaves quite differently from the native command: head -2 /dev/urandom returns instantly, the rewritten form never returns.

What is already fixed

The non-UTF-8 content loss described in the original version of this issue is fixed on develop by #3941 (merged as cfe248c1). read::run now reads bytes and applies the window on byte offsets when filtering and line numbering are off. Verified against the develop binary:

$ printf '\xff\xfe bad\nline2\nline3\n' > bin.log
$ head -2 bin.log                       → 13 bytes, rc=0
$ rtk read bin.log --head-lines 2       → 13 bytes, rc=0   byte-identical

That holds for head -N, head -n N, head --lines[=]N, bare head, and the tail equivalents. This issue is narrowed to what that change deliberately left open.

1. Unbounded streams never terminate

$ head -2 /dev/urandom                  # returns instantly
$ rtk read /dev/urandom --head-lines 2  # no output, killed at 5s

head -2 /dev/urandom is rewritten today. read/read_to_end has no line budget, so it reads until an EOF that never arrives.

Note for whoever picks this up: a FIFO whose writer closes is not affected — rtk read fifo --head-lines 1 completes normally, because EOF does arrive. The failure is specific to inputs with no EOF (character devices, yes |, a FIFO held open by a live writer), not to non-regular files as a category. The original title said "FIFOs"; that was too broad.

2. Peak memory is proportional to input size, not window size

On a 143 MB file, head -n 5, measured against develop @ cfe248c1:

#3941 improved this (the byte path removed a String round-trip, so the same case was 293 MB before it), but the whole file is still resident to emit five lines.

Suggested direction

Read incrementally and stop once the window is satisfied. head_window already scans forward for the Nth \n, so a buffered reader that stops there closes both (1) and (2) for the --head-lines case. --tail-lines needs the tail kept in a ring buffer rather than the whole input. Falling back to the native binary when the input is not a regular file would close (1) alone.

Related

• #3941 — fixed the content-loss half; merged.
• #3339 — savings accounting on these windows, including the from_utf8_lossy inflation that shares this code path.

Environment

• develop @ cfe248c1, Linux, LC_ALL=C

Start at the Rust read::run/read_to_end path and inspect the existing head_window scan. Reproduce the /dev/urandom and large-file commands, then verify that head stops at its window and tail uses bounded buffering; completion should avoid waiting for EOF and memory should scale with the requested window.